This is the abridged developer documentation for Swiftide # LLM applications in Rust: swift to build, built to ship. > An agent harness, typed task graphs and streaming RAG pipelines that work together, with retries, streaming, human approval, caching and tracing already there. [Get started](/start/what-is-swiftide/) Add swiftide to a project ```sh cargo add swiftide --features swiftide-agents,openai ``` [API reference](https://docs.rs/swiftide/0.34/swiftide/) · [GitHub](https://github.com/bosun-ai/swiftide) · [Discord](https://discord.gg/3jjXYen9UY) · [llms.txt](/llms.txt) · [For coding agents](/start/for-coding-agents/) $release-notes-agent --since v0.33.0 --version 0.34.0 ``` → git_log_since v0.33.0 git_log_since 346 lines → git_diff swiftide-integrations/Cargo.toml 3b1ad22e^..3b1ad22e → git_diff …/src/chat_completion/chat_completion_response.rs 90bcadc7^..90bcadc7 → git_diff swiftide-agents-contrib/Cargo.toml 71fa4587^..71fa4587 git_diff 15 lines git_diff 72 lines git_diff 75 lines → write_changelog <32 lines> write_changelog ⏸ waiting for approval Write these release notes? [y/N] y write_changelog wrote RELEASE_NOTES.md → stop (version 0.34.0: highlights 2, breaking 1, fixes 1) stop ■ done ``` examples/release-notes-agent/src/agent.rs ```rust /// Builds the agent. All tools run through `executor`, whose working directory is the repo. pub fn build( llm: &(impl ChatCompletion + Clone + 'static), executor: Arc, task: &ReleaseTask, github_tools: Option, ) -> Result { let mut builder = Agent::builder(); builder .llm(llm) .context(DefaultContext::from_executor(executor)) .system_prompt(task.system_prompt(github_tools.is_some())) // Our own `stop` tool replaces the default one; call this before `tools`. .without_default_stop_tool() .tools([ git_log_since(), git_diff(), read_file(), // Drafts that describe a change twice go back to the model before anyone reviews them. OneEntryPerChange::new(ApprovalRequired::new(WriteChangelog::new(&task.notes_file))) .into(), ReleaseNotes::stop_tool().into(), ]) .limit(STEP_LIMIT); if let Some(github_tools) = github_tools { builder.add_toolbox(github_tools); } progress::report_tool_calls(&mut builder); progress::stream_draft(&mut builder); Ok(builder.build()?) } ``` ## [Release notes agent](/showcases/release-notes-agent/) Reads git, drafts the notes, and writes them only after you say yes. `→ stop (version 0.34.0: highlights 2, breaking 1, fixes 1)` [Agents guide →](/agents/overview/) examples/workspace-architect/src/graph.rs ```rust /// Builds the task: `WorkspaceRoot` in, the written ARCHITECTURE.md out. pub fn architect_task( llm: &L, settings: ArchitectSettings, progress: &Progress, ) -> Result, TaskError> where L: ChatCompletion + SimplePrompt + Clone + 'static, { let mut task = Task::new(); let discover = task.register_node_fn({ let progress = progress.clone(); move |root: &WorkspaceRoot| { let members = workspace::discover(root, settings.selection)?; progress.step("discover", format_args!("{members} in {root}")); Ok::<_, workspace::DiscoveryError>(members) } }); let analyse = task.register_node(CrateAnalyst::new(llm.clone(), progress.clone())); let join = task.register_node_fn({ let progress = progress.clone(); move |summaries: &JoinInput| { progress.joined(summaries.len()); architecture_prompt(summaries) } }); // `Arc` is a task node out of the box: `Prompt` in, `String` out. let render = task.register_node(Arc::new(llm.clone()) as Arc); let review = task.register_node(ApprovalGate::new(settings.pre_approved, progress.clone())); let write = task.register_node_async_fn({ let output = settings.output; let progress = progress.clone(); move |draft: &ArchitectureDraft| { let output = output.clone(); let progress = progress.clone(); Box::pin(async move { let written = draft.write_to(&output).await?; progress.step( "write", format_args!("{} ({} bytes)", written.path.display(), written.bytes), ); Ok::<_, std::io::Error>(written) }) } }); task.starts_with(discover); // One branch per crate, all evaluating the same analyst node with a different input. task.register_transition(discover, move |members: WorkspaceMembers| { let mut crates = members.crates.into_iter(); let Some(first) = crates.next() else { return Transition::error("the workspace has no crates to document"); }; crates .fold(Transition::fan_out(&analyse, first), |fan_out, member| { fan_out.and(&analyse, member) }) .concurrency_model(ConcurrencyModel::Parallel) .join_with(join.join()) })?; // Each branch hands its `CrateSummary` to the join; the join runs once all have arrived. task.register_transition(analyse, join.join())?; task.register_transition(join, { let progress = progress.clone(); move |prompt| { progress.step("render", "drafting ARCHITECTURE.md"); render.transitions_with(prompt) } })?; task.register_transition(render, move |markdown: String| { review.transitions_with(ArchitectureDraft::from_model_output(&markdown)) })?; task.register_transition(review, move |decision| match decision { Review::Approved(draft) => write.transitions_with(draft).into(), Review::AwaitingApproval => Transition::pause(), })?; task.register_transition(write, task.transitions_to_finish())?; Ok(task) } ``` ## [Workspace architect](/showcases/workspace-architect/) One agent per crate, run in parallel and joined into ARCHITECTURE.md. `[ 40.5s] join 10 summaries; 230s of agent work in 41s wall-clock (5.7x)` [Tasks guide →](/tasks/overview/) examples/ask-your-codebase/src/index.rs ```rust let stats = indexing::Pipeline::from_loader( FileLoader::new(codebase.root()).with_extensions(&languages.extensions()), ) .filter(test_code::not_test_files(codebase.clone())) // Drop unit tests before the cache, which keys on path and content. Changing the content // after it would make the cache miss on every run. .then(test_code::without_unit_tests) .then(files_found.count()) // Skip files whose path and content are unchanged since the last successful run .filter_cached(codebase.node_cache()?) .then(files_changed.count()) .then(record_relative_path(codebase.clone())) // Split along syntax boundaries (functions, impls, ...) with tree-sitter .then_chunk(ChunkByLanguage::try_new(languages, CHUNK_SIZE)?) // Ask the LLM which questions each chunk answers; stored as metadata and embedded .then( MetadataQACode::from_client(llm) .prompt_template(QUESTIONS_PROMPT) .num_questions(QUESTIONS_PER_CHUNK) .concurrency(concurrent_requests) .build()?, ) // Embed chunk + metadata locally, in batches .then_in_batch(Embed::new(embedding::local_model()?).with_batch_size(embedding::BATCH_SIZE)) .then_store_with(codebase.vector_store()?) .run() .await?; ``` ## [Ask your codebase](/showcases/ask-your-codebase/) Indexes a codebase as a stream; unchanged files hit the cache. `Indexed 0 chunks from 0 new or changed files (14 files found, 14 unchanged) in 0.6s` [Pipelines guide →](/indexing/overview/) ## The same parts, three places The same client, executor or store works in an agent, a task graph and a pipeline, so you wire a value once. * **Values**\ an `OpenAI` client, a `LocalExecutor`, a `Qdrant` store * **Traits**\ `ChatCompletion`, `SimplePrompt`, `ToolExecutor`, `Persist`, `Retrieve` * **Runtimes**\ [an agent](/agents/overview/), [a task graph](/tasks/overview/), [a pipeline](/indexing/overview/) ## Integrations * [openai](/integrations/openai/) * [anthropic](/integrations/anthropic/) * [aws-bedrock](/integrations/aws-bedrock/) * [ollama](/integrations/openai-compatible/) * [fastembed](/integrations/fastembed/) * [qdrant](/integrations/qdrant/) * [lancedb](/integrations/lancedb/) * [pgvector](/integrations/pgvector/) * [redis](/indexing/caching/) * [tree-sitter](/integrations/tree-sitter/) * [mcp](/agents/mcp/) * [All 24 →](/integrations/overview/) # Using Swiftide with coding agents > Give Claude Code, Cursor, Codex and other coding agents the Swiftide skill, Context7, llms.txt, per-page Markdown, rustdoc JSON and an AGENTS.md block pinned to the right version. Your coding agent, such as Claude Code, Cursor or Codex, can write Swiftide code that compiles against 0.34. Swiftide is pre-1.0, and minor releases break APIs. Release 0.33 changed system prompts, hook signatures, command output and the AWS Bedrock module. A model that learned an older release writes code that no longer compiles. Each source below gives the agent the current API, and the [AGENTS.md block](#agentsmd-block) at the end pins the rules in your project. | Source | What the agent gets | | -------------- | --------------------------------------------------------- | | Skill | Rules, gotchas and compiled snippets for the current API | | Context7 | Search over the examples, the skill and the crate sources | | llms.txt files | This site as plain text | | Page Markdown | One page as Markdown | | API docs | Exact signatures, as HTML or rustdoc JSON | | Examples | Programs that compile | ## Release status The skill, `context7.json` and the four showcase crates ship with the 0.34 release. These docs pin the release commit, and the skill describes the APIs at that commit. ## Skill The Swiftide skill is a set of instructions that the agent loads when the task needs it. Install it with the `skills` CLI from npm (needs Node.js): ```sh npx skills add bosun-ai/swiftide ``` Without Node.js, copy the [`skills/swiftide`](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/skills/swiftide) folder from the pinned commit into the skills directory of your agent. `SKILL.md` covers feature flags, OpenAI-compatible endpoints, tools, hooks, structured stops, human approval, MCP (Model Context Protocol) toolboxes, tasks and pipelines, and ends with a checklist. Four reference files go deeper: `agents.md`, `tasks.md`, `pipelines.md` and `integrations.md`. Every snippet in them is copied from compiled code in the swiftide repository, and names its source file. The agent picks the skill up when a Rust project depends on swiftide or one of its crates, when you ask for an agent, task graph or RAG (retrieval-augmented generation) pipeline in Rust, or when it fixes a missing feature flag. ## Context7 [Context7](https://context7.com) indexes the swiftide repository under the library ID `/bosun-ai/swiftide`. With the Context7 MCP server installed, ask the agent to look up Swiftide in Context7, or give it the ID. Context7 indexes the default branch, which can hold APIs that are not released yet. Check what it returns against the version in your `Cargo.lock`. The repository’s [`context7.json`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/context7.json) limits the index to `examples`, `skills` and the source folders of the crates, and adds nine rules that the AGENTS.md block below also covers. ## llms.txt files The site publishes plain-text versions of itself for language models. `/llms.txt` states the version these docs describe, 0.34, and links the others. | File | Contents | | ------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------- | | [/llms.txt](https://swiftide.rs/llms.txt) | An index with the pinned version and links to the files below | | [/llms-full.txt](https://swiftide.rs/llms-full.txt) | Every page in one file | | [/llms-small.txt](https://swiftide.rs/llms-small.txt) | The start pages and core guides, without recipes, showcases, integrations or reference | | [/\_llms-txt/agents.txt](https://swiftide.rs/_llms-txt/agents.txt) | Agents: tools, hooks and executors | | [/\_llms-txt/tasks.txt](https://swiftide.rs/_llms-txt/tasks.txt) | Typed task graphs | | [/\_llms-txt/indexing-and-querying.txt](https://swiftide.rs/_llms-txt/indexing-and-querying.txt) | Indexing and query pipelines | | [/\_llms-txt/integrations.txt](https://swiftide.rs/_llms-txt/integrations.txt) | Model providers, vector stores and other integrations | | [/\_llms-txt/showcases.txt](https://swiftide.rs/_llms-txt/showcases.txt) | The four showcase programs and their output | The full and small files start with the home page, followed by the Start pages. None of the files include the changelog. A section file is the smallest useful input when the agent works on one part, such as a task graph. ## Page Markdown Every page is also served as Markdown at its URL plus `.md`, for example . Snippets are expanded into code, so the agent sees the same compiled Rust as you do. The buttons next to each page title do the same by hand. **Copy page** copies the page as Markdown, for pasting into a chat. **View as Markdown** opens the `.md` version. **Open in ChatGPT** and **Open in Claude** start a conversation about the page. ## API docs The [API docs on docs.rs](https://docs.rs/swiftide/0.34/swiftide/index.html) are the most reliable source of exact signatures. docs.rs builds swiftide with every feature flag on and labels each item with the flag it needs. docs.rs also serves rustdoc JSON. The `swiftide` crate only re-exports the other crates, so its JSON holds re-export stubs and no signatures. Fetch the JSON of the crate that defines the items, such as `swiftide-agents`, `swiftide-tasks`, `swiftide-core`, `swiftide-indexing`, `swiftide-query` or `swiftide-integrations`. docs.rs sends it zstd-compressed, so decompress it before the agent reads it: ```sh curl -sSL https://docs.rs/crate/swiftide-agents/0.34/json | zstd -d > swiftide-agents.json ``` Pin the version in the URL so the agent does not read a newer release. `latest` in place of the version gives the newest release instead. ## Examples The [examples directory](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples) holds the single-file examples and the four [showcases](/showcases/release-notes-agent/). Each showcase has a README, an `OUTPUT.md` with output from a real run, and source that this site excerpts. From the root of a checkout of the pinned commit, an agent can run them. `hello-agents` uses Gemini and needs `GEMINI_API_KEY`. `ask-your-codebase` needs `OPENAI_API_KEY`, and its first run downloads an embedding model of about 600 MB. ```sh cargo run -p swiftide-examples --example hello-agents cargo run -p ask-your-codebase -- index swiftide-agents ``` ## AGENTS.md block Paste this block into the `AGENTS.md` or `CLAUDE.md` at the root of your project. It comes from the skill’s gotchas and the rules in `context7.json`. Change the version line when you upgrade. ```md ## Swiftide This project uses swiftide 0.34 (Rust 1.97 or newer, edition 2024). Swiftide is pre-1.0 and minor releases break APIs. Check the version in Cargo.lock, and read https://swiftide.rs/llms.txt or https://docs.rs/swiftide/0.34/swiftide/ before trusting examples written for older releases. - Feature flags: default features are only `macros` (`#[swiftide::tool]`, `#[derive(Tool)]`). Agents need `swiftide-agents`, which also turns on `swiftide-tasks`. Each integration needs its own flag (`openai`, `anthropic`, `gemini`, `qdrant`, `fastembed`, `lancedb`, `duckdb`, `redb`, `mcp`, ...). An unresolved import under `swiftide::agents` or `swiftide::integrations` means a missing flag. `all` turns on only `qdrant`, `redis`, `tree-sitter`, `openai`, `fastembed`, `scraping`, `aws-bedrock`, `groq`, `ollama` and `pgvector`; everything else, including agents and tasks, needs its own flag. - With `default-features = false`, add `macros` back. Tools and structured stops also need `schemars` (with `derive`) and `serde`. - TLS needs no setup: the HTTP clients use rustls with their own crypto provider. Do not add `rustls` or call `install_default()` unless the app wants a specific provider. - Tools: `#[swiftide::tool]` on an async fn that takes `&dyn AgentContext` and returns `Result`. The doc comment is the tool description and a `# Arguments` list describes the arguments. Use `#[derive(Tool)]` for a tool with state. Return `ToolOutput::fail(..)` for errors the model should see and recover from. - Run commands with `context.executor().exec_cmd(&Command::shell(..))`; `AgentContext::exec_cmd` is deprecated. `CommandOutput` records stdout and stderr chunks: `stdout()` and `stderr()` iterate over the `Bytes` chunks of one stream, and `into_string_lossy()` merges both streams into one `String` in the order they arrived. - The system prompt is a `Prompt`: `.system_prompt(..)` takes anything `Into`, such as a `&str`. `SystemPrompt` no longer exists. An agent without a system prompt sends none. - Hooks are closures that return `Box::pin(async move { Ok(()) })`; clone captured values before the `async move` block. Most hooks get `&mut Agent`; `on_new_message` and `on_stream` get `&Agent`. A hook that returns an error is logged, not propagated. - Structured output: call `without_default_stop_tool()` before `.tools(..)`, add `StopWithArgs::with_parameters_schema(schema_for!(T))`, and read `StopReason::RequestedByTool(_, Some(payload))`. `.tools(..)` replaces any earlier list. - `StopReason::NoNewMessages` means the model answered without a tool call or `.limit(n)` was reached. `limit` counts completions per `query`. - Human approval: wrap a tool in `ApprovalRequired::new(tool)`. On `StopReason::FeedbackRequired`, call `agent.context().feedback_received(&tool_call, &ToolFeedback::approved())`, then `agent.run()`. - Tasks: import from `swiftide::tasks`; `swiftide::agents::tasks` is deprecated. Use `Task`, `TaskNode` and `Transition` instead of chaining agents by hand. Give every node one transition and end with `task.transitions_to_finish()`. A fan-out (`Transition::fan_out(..).and(..).join_with(join.join())`) ends in a join node. `task.run(..)` returns `TaskRunOutcome::Completed(output)` or `TaskRunOutcome::Paused`; continue a paused task with `task.resume()`. - OpenAI-compatible endpoints go through the `openai` flag: `OpenAI::builder().api_base("http://127.0.0.1:8080/v1").api_key(key)`. The URL includes the version prefix. `.client(..)` replaces both; whichever is called last wins. - AWS Bedrock is `swiftide::integrations::aws_bedrock_v2` (the Converse API), behind the `aws-bedrock` flag. - MCP tools are named `:`. OpenAI rejects `:` in tool names, so rename them. - Pipelines: use the same embedding model for indexing and querying, and set the vector store's `vector_size` to that model's dimensions. Wrap LLM clients used during indexing in `LanguageModelWithBackOff`. - Before finishing: run `cargo check` with the exact features. ``` ## Verification Ask the agent to finish with the checks from the skill. `cargo check` with the exact feature flags in `Cargo.toml` catches a missing flag as an unresolved import (error E0432). ## Reference * [`swiftide` crate root](https://docs.rs/swiftide/0.34/swiftide/index.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) and [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) and [`CommandOutput`](https://docs.rs/swiftide/0.34/swiftide/traits/struct.CommandOutput.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`ApprovalRequired`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.ApprovalRequired.html) and [`StopWithArgs`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.StopWithArgs.html) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html), [`Transition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html) and [`TaskRunOutcome`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskRunOutcome.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) and [`OpenAIBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAIBuilder.html) * [`aws_bedrock_v2`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/index.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`skills/swiftide/SKILL.md`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/skills/swiftide/SKILL.md) at the pinned commit * [Feature flags](/reference/feature-flags/) * [Upgrading](/reference/upgrading/) * [TLS](/reference/feature-flags/#tls) ## Next [Agents overview](/agents/overview/) # Installation > Add swiftide with the feature flags you need and tokio, set your API keys, and run a first program that prints one answer from a model. You end with a Rust binary on swiftide 0.34 that prints one answer from a model. The page also lists the environment variables, feature flags and local servers the rest of the site uses. ## Dependencies Swiftide 0.34 needs Rust 1.97 or newer, its minimum supported Rust version (MSRV). Create a binary crate and add three dependencies: ```sh cargo new hello-swiftide cd hello-swiftide cargo add swiftide@0.34 --features swiftide-agents,openai cargo add tokio --features macros,rt-multi-thread cargo add anyhow ``` `openai` compiles the OpenAI client. `swiftide-agents` is for [Your first agent](/start/your-first-agent/); this page does not use it yet. tokio runs the async `main`, and anyhow carries its errors. ## First program Replace `src/main.rs` with this program: src/main.rs ```rust use swiftide::{integrations::openai::OpenAI, traits::SimplePrompt}; #[tokio::main] pub async fn main() -> anyhow::Result<()> { // Reads OPENAI_API_KEY from the environment. let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .build()?; let answer = openai .prompt("Say hello to Swiftide in five words.".into()) .await?; println!("{answer}"); Ok(()) } ``` `OpenAI::builder()` without a `client(..)` call reads `OPENAI_API_KEY` from the environment, and `OPENAI_BASE_URL` when it is set. `default_prompt_model` sets the model that `prompt` uses. `prompt` comes from the `SimplePrompt` trait, which the `use` line imports. It takes a `Prompt`, which `.into()` builds from a string. ### TLS Swiftide’s Hypertext Transfer Protocol (HTTP) clients use rustls for Transport Layer Security (TLS), with the HTTP client’s own crypto provider, so the program needs no TLS setup. A provider that your application installs for the whole process takes precedence. [TLS](/reference/feature-flags/#tls) covers the one exception, Redis over `rediss://`. ## API keys Export your application programming interface (API) key and run the program: ```sh export OPENAI_API_KEY=sk-... cargo run ``` Cargo builds the crate and the program prints the model’s one-line answer. These environment variables come up across the site and the showcases: | Variable | Read by | Default | | ------------------------------------------------ | --------------------------------------------- | ----------------------------------------------- | | `OPENAI_API_KEY` | the OpenAI client (`openai` flag) | none | | `OPENAI_BASE_URL` | the OpenAI client (`openai` flag) | `https://api.openai.com/v1` | | `OPENAI_MODEL` | the showcase command-line tools, not Swiftide | `gpt-5.4` or `gpt-5.4-mini` | | `QDRANT_URL`, `QDRANT_API_KEY` | the Qdrant builder (`qdrant` flag) | `http://localhost:6334` | | `GROQ_API_KEY` | the Groq client (`groq` flag) | none | | `GEMINI_API_KEY` | the Gemini client (`gemini` flag) | none | | `OPENROUTER_API_KEY` | the OpenRouter client (`open-router` flag) | none | | `QWEN_API_KEY`, then `DASHSCOPE_API_KEY` | the DashScope client (`dashscope` flag) | none | | `GITHUB_PERSONAL_ACCESS_TOKEN` or `GITHUB_TOKEN` | release-notes-agent with `--mcp` | none | | `LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY` | the Langfuse tracing layer (`langfuse` flag) | required; the default layer panics without them | | `LANGFUSE_URL` | the Langfuse tracing layer (`langfuse` flag) | `http://localhost:3000` | The Langfuse layer also accepts `LANGFUSE_INIT_PROJECT_PUBLIC_KEY` and `LANGFUSE_INIT_PROJECT_SECRET_KEY` as fallbacks for the two keys. You can point `OPENAI_BASE_URL` at an OpenAI-compatible endpoint, such as a local gateway. The endpoint must serve the model name you pass to `default_prompt_model`, here `gpt-5-mini`. Unset the variable rather than leave it empty: an empty value becomes an empty base URL. [OpenAI-compatible providers](/integrations/openai-compatible/) and [Models](/models/overview/) cover the other clients. ## Feature flags Only `macros` is on by default. It provides the `#[tool]` attribute, `#[derive(Tool)]` and `#[indexing_transformer]`. Agents, tasks and each integration sit behind their own feature flag, and each flag adds a module: * `openai` adds `swiftide::integrations::openai`. * `qdrant` adds `swiftide::integrations::qdrant`. * `swiftide-agents` adds `swiftide::agents` and turns on `swiftide-tasks`, which adds `swiftide::tasks`. The showcases name only the flags they use: | Showcase | Flags | | ------------------------------------------------------ | -------------------------------------------------------- | | [release-notes-agent](/showcases/release-notes-agent/) | `macros`, `swiftide-agents`, `openai`, `mcp`, `langfuse` | | [ask-your-codebase](/showcases/ask-your-codebase/) | `tree-sitter`, `fastembed`, `lancedb`, `redb`, `openai` | | [workspace-architect](/showcases/workspace-architect/) | `swiftide-agents`, `swiftide-tasks`, `openai` | | [docs-qa](/showcases/docs-qa/) | `openai`, `fastembed`, `qdrant`, `langfuse` | Three of them turn default features off. With `--no-default-features`, add `macros` back if you use any of its macros. The `all` flag is not a shortcut to everything: it turns on only `qdrant`, `redis`, `tree-sitter`, `openai`, `fastembed`, `scraping`, `aws-bedrock`, `groq`, `ollama` and `pgvector`. [Feature flags](/reference/feature-flags/) lists every flag and what it turns on. ## Local services LanceDB, DuckDB, FastEmbed and Redb run inside your process, so the ask-your-codebase showcase needs no server. FastEmbed downloads its model on first use. Swiftide’s FastEmbed default is the fastembed crate’s default model; ask-your-codebase picks `jina-embeddings-v2-base-code` instead, which is about 600MB. Qdrant, Redis and PgVector run as servers. Docker starts each one: ```sh # Qdrant (qdrant flag): Swiftide talks gRPC Remote Procedure Call (gRPC) on 6334; # 6333 is the HTTP API and dashboard. docker run -d -p 6333:6333 -p 6334:6334 qdrant/qdrant # Redis (redis flag): the examples read REDIS_URL and default to redis://localhost:6379. docker run -d -p 6379:6379 redis:7-alpine # PgVector (pgvector flag): pass the connection URL to the PgVector builder with db_url. docker run -d -p 5432:5432 -e POSTGRES_PASSWORD=postgres pgvector/pgvector:pg17 ``` ## Common errors * **The program panics with “No rustls crypto provider is configured”**: you are on 0.33 or earlier. See [Crypto provider panic](/reference/troubleshooting-and-faq/#crypto-provider-panic). * **Error E0432, “unresolved import”**: the feature flag behind that module is off. Add it with `cargo add swiftide@0.34 --features `. * **An authentication error (HTTP 401)**: `OPENAI_API_KEY` is missing or empty. The client builds without a key, so the error shows up at the first request. [Troubleshooting and FAQ](/reference/troubleshooting-and-faq/) covers more. ## Reference * [`swiftide` crate root, Cargo Features section](https://docs.rs/swiftide/0.34/swiftide/index.html#cargo-features) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`GenericOpenAIBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) ## Next [Your first agent](/start/your-first-agent/) # What is Swiftide > Swiftide is a Rust library with three building blocks, agents, tasks, and indexing and query pipelines, and this page shows each one in a real program and when to pick it. Swiftide is a Rust library for building LLM (large language model) applications. It has three building blocks: agents for tool loops, tasks for typed hand-offs, and pipelines for data flows. This page shows each block in a real program, so you know which one to start with. ## Building blocks | Building block | Use it for | Entry point | Feature flag | | -------------- | ------------------------------------------------------------ | ----------------------------------------------------------- | ----------------------------------------------------- | | Agents | A model that calls tools in a loop until it stops | `swiftide::agents::Agent::builder()` | `swiftide-agents` | | Tasks | A typed graph of steps with fan-out, join and pause | `swiftide::tasks::Task::new()` | `swiftide-tasks`, also turned on by `swiftide-agents` | | Pipelines | Streaming data into storage, and answering questions from it | `swiftide::indexing::Pipeline`, `swiftide::query::Pipeline` | always compiled; each integration has its own flag | The three blocks share one set of core traits, such as `SimplePrompt`, `ChatCompletion`, `EmbeddingModel` and `Tool`. The `OpenAI` client implements the first three, so one client can serve all three blocks. It needs the `openai` feature flag. ### Agents ![Agent loop](/diagrams/diagram-agent-loop.svg) The release-notes-agent showcase reads the git history since the last tag and drafts release notes. This function builds its agent: examples/release-notes-agent/src/agent.rs ```rust /// Builds the agent. All tools run through `executor`, whose working directory is the repo. pub fn build( llm: &(impl ChatCompletion + Clone + 'static), executor: Arc, task: &ReleaseTask, github_tools: Option, ) -> Result { let mut builder = Agent::builder(); builder .llm(llm) .context(DefaultContext::from_executor(executor)) .system_prompt(task.system_prompt(github_tools.is_some())) // Our own `stop` tool replaces the default one; call this before `tools`. .without_default_stop_tool() .tools([ git_log_since(), git_diff(), read_file(), // Drafts that describe a change twice go back to the model before anyone reviews them. OneEntryPerChange::new(ApprovalRequired::new(WriteChangelog::new(&task.notes_file))) .into(), ReleaseNotes::stop_tool().into(), ]) .limit(STEP_LIMIT); if let Some(github_tools) = github_tools { builder.add_toolbox(github_tools); } progress::report_tool_calls(&mut builder); progress::stream_draft(&mut builder); Ok(builder.build()?) } ``` The agent sends its message history to the model, calls the tools the model asks for, and adds their results to the history. The tools run their commands through the tool executor in the context, here a `DefaultContext` built from the repository’s executor. `progress::report_tool_calls` prints each tool call from `before_tool` and `after_tool` hooks. `progress::stream_draft` registers an `on_stream` hook, which switches the agent to streaming completions. `add_toolbox` adds optional GitHub tools from an MCP (Model Context Protocol) server, which needs the `mcp` feature flag. The loop ends when a tool asks the agent to stop, when a tool needs human feedback (here `ApprovalRequired` around `WriteChangelog`), when the model replies without calling a tool, or when it reaches the step limit. Each case sets a stop reason. Reaching the limit is not an error, so check the stop reason after `query` returns. This agent swaps the default `stop` tool for its own, which requires the full release notes as structured arguments. `limit(STEP_LIMIT)` caps the completions per query at 40. [Your first agent](/start/your-first-agent/) builds the first step of this agent with one tool and one hook. Read more in [Agents overview](/agents/overview/) and [Release notes agent](/showcases/release-notes-agent/). ### Tasks ![Task graph](/diagrams/diagram-task-graph.svg) The workspace-architect showcase drafts an `ARCHITECTURE.md` for a Cargo workspace. Its task first discovers the member crates, then fans out to one analyst per crate: examples/workspace-architect/src/graph.rs ```rust // One branch per crate, all evaluating the same analyst node with a different input. task.register_transition(discover, move |members: WorkspaceMembers| { let mut crates = members.crates.into_iter(); let Some(first) = crates.next() else { return Transition::error("the workspace has no crates to document"); }; crates .fold(Transition::fan_out(&analyse, first), |fan_out, member| { fan_out.and(&analyse, member) }) .concurrency_model(ConcurrencyModel::Parallel) .join_with(join.join()) })?; ``` Every task node has an input type and an output type. The compiler checks that each transition hands the next node the input it expects. The transition after `discover` starts one branch per crate on the same `analyse` node and runs the branches with `ConcurrencyModel::Parallel`. `join_with` names the join node that collects the results. The `analyse` node still needs its own transition into the join, `task.register_transition(analyse, join.join())`, or the task fails validation with a missing-transition error. An empty workspace ends the task with `Transition::error`. A run on the swiftide workspace with claude-sonnet-5-5, through an OpenAI-compatible gateway, prints this (trimmed): ```text [ 0.0s] analyse started swiftide-core [ 0.0s] analyse started swiftide-langfuse ... [ 40.5s] analyse finished swiftide-core in 40.5s (15 tool calls) [ 40.5s] join 10 summaries; 230s of agent work in 41s wall-clock (5.7x) ``` [Your first task](/start/your-first-task/) builds a smaller task that starts the same way, with a `discover` node and no fan-out. Read more in [How tasks work](/tasks/overview/) and [Workspace architect](/showcases/workspace-architect/). ### Pipelines ![Indexing pipeline](/diagrams/diagram-pipeline.svg) An indexing pipeline streams nodes from a loader through transformers, a chunker and an embedder into storage. A query pipeline turns a question into an answer from retrieved chunks. The ask-your-codebase showcase indexes a codebase into LanceDB and answers questions about it with this query pipeline: examples/ask-your-codebase/src/query.rs ```rust let search = SimilaritySingleEmbedding::default().with_top_k(top_k); let pipeline = query::Pipeline::from_search_strategy(search) // Rephrase the question into several, to widen what the search can match .then_transform_query( GenerateSubquestions::builder() .client(llm.clone()) .prompt_template(SUBQUESTIONS_PROMPT.into()) .build()?, ) // Embed with the same local model that embedded the chunks .then_transform_query(Embed::from_client(embedding::local_model()?)) .then_retrieve(codebase.vector_store()?) // Answer the original question from the retrieved code, each chunk under its file path .then_answer( answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(CHUNK_TEMPLATE) .build()?, ); let answered = pipeline.query(question).await?; ``` Query transformers rewrite and embed the question, the retriever searches the LanceDB table with the search strategy, and the answer generator writes the answer from the retrieved chunks. [Your first pipeline](/start/your-first-pipeline/) runs this showcase and walks through each step, along with the indexing pipeline that fills the table. The showcase needs the `lancedb`, `fastembed`, `tree-sitter` and `openai` feature flags. The docs-qa showcase adds hybrid search in Qdrant (the `qdrant` feature flag), reranking and evaluation. Read more in [Indexing overview](/indexing/overview/), [How querying works](/querying/overview/), [Ask your codebase](/showcases/ask-your-codebase/) and [Docs Q\&A](/showcases/docs-qa/). ## Composition The blocks nest. In workspace-architect, every fan-out branch runs an agent. `CrateAnalyst` implements `TaskNode` with a `WorkspaceCrate` in and a `CrateSummary` out. Each evaluation builds a fresh agent for one crate: examples/workspace-architect/src/analyst.rs ```rust /// How often an agent is told to fix a missing or malformed report before the crate fails. const MAX_CORRECTIONS: usize = 2; #[async_trait] impl TaskNode for CrateAnalyst { type Input = WorkspaceCrate; type Output = CrateSummary; type Error = AnalystError; async fn evaluate( &self, _node_id: &DynNodeId, member: &WorkspaceCrate, ) -> Result { self.analyse(member).await.map_err(|cause| AnalystError { crate_name: member.name.clone(), cause, }) } } impl CrateAnalyst { async fn analyse(&self, member: &WorkspaceCrate) -> Result { let analysis = self.progress.analysis(&member.name); let mut agent = self.agent_for(member, analysis.clone())?; agent.query(brief(member)).await?; // Models occasionally answer in prose or nest the `stop` arguments; say what went wrong. let mut corrections = 0; let report = loop { match submitted_report(&agent) { Ok(report) => break report, Err(problem) if corrections < MAX_CORRECTIONS => { corrections += 1; analysis.corrected(&problem); agent.query(problem.correction()).await?; } Err(problem) => return Err(problem.into()), } }; analysis.finished(&report); Ok(report.reconciled_with(member)) } } /// The report the agent passed to `stop`, checked against the `CrateReport` schema. fn submitted_report(agent: &Agent) -> Result { match agent.stop_reason() { Some(StopReason::RequestedByTool(_, Some(arguments))) => { Ok(CrateReport::deserialize(arguments)?) } other => Err(ReportProblem::Missing { stop_reason: other.cloned(), }), } } ``` Two built-in node types need no impl of your own. `Arc` takes a `Prompt` and returns a `String`; the showcase’s render step is one. `TaskAgent` wraps an `Agent` as a node that takes a `Prompt` and returns `()`. To pass an answer to the next node, write your own `TaskNode`. `TaskAgent` also locks the agent, so branches that share one `TaskAgent` run one at a time. For parallel work, build one agent per branch as `CrateAnalyst` does. A task node or a tool is plain async Rust, so its body can run an indexing pipeline or a query pipeline. ## Block selection | You need | Use | | ------------------------------------------------------------------------------------------- | ----------------------------------------- | | A model that decides which tools to call and when to stop | an agent | | Known steps with typed inputs and outputs, parallel branches, or a pause for human approval | a task | | Documents or code loaded, chunked, embedded and stored, then searched to answer questions | an indexing pipeline and a query pipeline | | Several agents in a fixed order, or one agent per item in parallel | a task with agent nodes | Start with an agent when the model should choose the steps. Move to a task when you know the steps and want the compiler to check the hand-offs between them. ## Project status Swiftide is pre-1.0. APIs change while the agent and task APIs settle, and breaking changes land in minor releases. Upgrade one minor version at a time with [Upgrading](/reference/upgrading/), and check the [Changelog](/reference/changelog/) for each release. These docs describe swiftide 0.34. Swiftide is MIT licensed and part of the [bosun.ai](https://bosun.ai) project. ## Reference * [`swiftide`](https://docs.rs/swiftide/0.34/swiftide/index.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`TaskAgent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.TaskAgent.html) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html) * [`TaskNode`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.TaskNode.html) * [`Transition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html) * [`ConcurrencyModel`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.ConcurrencyModel.html) * [`indexing::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`GenerateSubquestions`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.GenerateSubquestions.html) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.Embed.html) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`Tool`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Tool.html) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [Feature flags](/reference/feature-flags/) ## Next [Installation](/start/installation/) # Your first agent > Build an agent with one git tool and one hook that summarizes the commits since a tag, then compare it with the release-notes-agent showcase. This page builds a command-line program that asks a model to summarize the commits since a git tag. It is the first step of the [release-notes-agent](/showcases/release-notes-agent/) showcase: an agent with one tool that runs `git log` and one hook that prints each tool call. ## Dependencies Start from the crate on [Installation](/start/installation/), or create a new one with the same dependencies plus serde: ```sh cargo new first-agent cd first-agent cargo add swiftide@0.34 --features swiftide-agents,openai cargo add tokio --features macros,rt-multi-thread cargo add anyhow serde ``` `swiftide-agents` adds `swiftide::agents`. The tool attribute comes from `macros`, which is on by default. Its generated code derives serde’s traits for the tool arguments, so your crate depends on serde. ## Program The tool and the imports go at the top of `src/main.rs`: src/main.rs ```rust use anyhow::Context as _; use swiftide::{ agents::Agent, chat_completion::{ChatMessage, ToolOutput, errors::ToolError}, integrations::openai::OpenAI, traits::{AgentContext, Command}, }; /// Lists the commits since a git tag, newest first, one per line. /// /// # Arguments /// - `tag`: The tag to start from, e.g. `v0.33.0`. #[swiftide::tool] async fn git_log_since(context: &dyn AgentContext, tag: &str) -> Result { // The model chooses the tag; refuse anything that is not a plain tag name. let plain = |c: char| c.is_ascii_alphanumeric() || "._-/".contains(c); if tag.starts_with('-') || !tag.chars().all(plain) { return Ok(ToolOutput::fail(format!("`{tag}` is not a tag name"))); } let log = Command::shell(format!("git log --oneline --end-of-options {tag}..HEAD")); Ok(context.executor().exec_cmd(&log).await?.into()) } ``` `main` builds the agent and prints its answer: src/main.rs ```rust #[tokio::main] pub async fn main() -> anyhow::Result<()> { let tag = std::env::args() .nth(1) .context("usage: first-agent ")?; // Reads OPENAI_API_KEY from the environment. let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .build()?; let mut agent = Agent::builder() .llm(&openai) .system_prompt("You summarize git history for release notes. Be brief.") .tools([git_log_since()]) .before_tool(|_agent, tool_call| { println!( " → {} {}", tool_call.name(), tool_call.args().unwrap_or_default() ); Box::pin(async { Ok(()) }) }) .build()?; agent .query(format!("Summarize the commits since {tag}.")) .await?; let history = agent.history().await?; let summary = history.iter().rev().find_map(|message| match message { ChatMessage::Assistant(Some(text), _) => Some(text.as_str()), _ => None, }); println!("{}", summary.unwrap_or("(no summary)")); println!("stopped: {:?}", agent.stop_reason()); Ok(()) } ``` The tool runs `git` in the directory you start the program from. That directory needs a git repository with the tag you pass. The repository `cargo new` created has no commits and no tags, so run the program from another repository, such as a clone of swiftide: ```sh export OPENAI_API_KEY=sk-... git clone https://github.com/bosun-ai/swiftide cd swiftide cargo run --manifest-path ../first-agent/Cargo.toml -- v0.33.0 ``` The program prints one line per tool call, such as `→ git_log_since {"tag":"v0.33.0"}`. Then it prints the model’s summary. The last line is `stopped: Some(NoNewMessages)` or `stopped: Some(RequestedByTool(..))`, because `stop_reason()` returns an `Option`. ## Tool `#[swiftide::tool]` turns the async function into a tool named `git_log_since`. The macro replaces your function: `git_log_since()` becomes a constructor that returns the tool as a `Box`, and your body moves to a method on a generated struct, so you cannot call the function directly any more. `.tools([git_log_since()])` hands the tool to the agent. The doc comment becomes the tool description the model reads. The `# Arguments` list becomes the description of each argument. Every tool takes `&dyn AgentContext` as its first argument. `context.executor()` returns the tool executor, the part that runs commands. `Command::shell` describes a shell command, and `exec_cmd` runs it. The model chooses `tag`, so the tool checks it before it reaches the shell. A tag that starts with `-`, or holds anything but letters, digits, `.`, `_`, `-` and `/`, gets `ToolOutput::fail`. The model gets that message as the tool’s output and can try again. `--end-of-options` also stops git from reading the tag as a flag. When git exits with a non-zero status, `exec_cmd` returns a `CommandError`, and `?` turns it into a `ToolError`. The agent also sends that error back to the model. When a call with the same arguments fails more than `tool_retry_limit` times (3 by default), the agent stops with `StopReason::ToolCallsOverLimit` and `query` returns the error. On success, `.into()` turns the `CommandOutput` into `ToolOutput::Text`, with stdout and stderr in the order the executor saw them. ## Hook A hook is a closure that runs at a fixed point in the agent loop. `before_tool` runs before each tool call and receives `&mut Agent` and the `ToolCall`. Every hook returns a boxed future, written `Box::pin(async { Ok(()) })`. When a hook returns an error, the agent logs it with tracing and carries on. ## Agent loop `Agent::builder()` starts with one tool already registered: `stop`, which the model calls when it is done. `.llm(&openai)` clones the client into the agent; `build` fails without it. `system_prompt` takes anything that converts into a `Prompt`, such as a `&str`. `query` adds your message to the message history and loops. Each completion either calls tools, whose outputs go into the history for the next completion, or answers in text. In a normal run the loop ends in one of two ways: * The model calls `stop`. `stop_reason()` then returns `StopReason::RequestedByTool`. * The model answers without calling a tool. The default context, `DefaultContext`, ends the loop on such an answer, and `stop_reason()` returns `StopReason::NoNewMessages`. Errors, failed tool calls and limits end it with other stop reasons. `history()` returns every `ChatMessage` of the run. The program prints the last `ChatMessage::Assistant` that has text. `DefaultContext` runs commands through a `LocalExecutor` in your working directory, without a sandbox: the tool can reach anything your user can. It sets no timeout either, so a git command that waits for a pager or a credential prompt blocks the agent. ## Showcase version The showcase prints its tool calls with the same kind of hook. This is the start of a real run on swiftide’s own history, from its `OUTPUT.md`. It used `claude-sonnet-5-5` through an OpenAI-compatible gateway: ```text Drafting release notes for the 7 commits since v0.33.0 with claude-sonnet-5-5 at http://127.0.0.1:8080/v1 (chat completions) → git_log_since v0.33.0 git_log_since ✓ 346 lines ``` It also adds the timeout your program lacks. Every command runs at the repository root, with a one-minute `default_timeout`: examples/release-notes-agent/src/main.rs ```rust let root = Git::new(&LocalExecutor::new(&cli.repo)) .top_level() .await .with_context(|| format!("{} is not a git repository", cli.repo.display()))?; // Every tool the agent runs, runs with the repository as its working directory. let executor: Arc = Arc::new( LocalExecutor::builder() .workdir(&root) .default_timeout(Some(Duration::from_mins(1))) .build()?, ); ``` The agent gets this executor through `DefaultContext::from_executor`. Use the same pattern to run your program against a fixed repository instead of the current directory. ## Variations To print results too, add `after_tool`. It receives `&mut Result` and can replace what the model sees. ## Common mistakes * **A second `.tools(..)` call drops the first list.** `.tools(..)` replaces the whole set with the default tools plus the ones you pass. Pass every tool in one call, or call `add_tool` after the last `.tools(..)`. * **Your own stop tool is dropped.** Tools are kept by name, and a typed stop tool such as `StopWithArgs` is also named `stop`. `.tools(..)` inserts the default `stop` first, so yours is discarded and the model sees only the default. Call `without_default_stop_tool()` before `.tools(..)`, as the showcase does. * **The model passes an argument you did not expect.** Check model-chosen arguments before they reach a shell or a path. * **The agent loops for a long time.** `.limit(n)` caps the completions per `query`; the showcase uses 40. ## Related pages * [Tools](/agents/tools/): `#[derive(Tool)]` for tools with state, and argument types. * [Hooks](/agents/hooks/): every hook and what it receives. * [Executors](/agents/executors/): `LocalExecutor`, timeouts and other executors. * [Stopping and structured results](/agents/stopping-and-structured-results/): a stop tool that returns typed data. ## Reference * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`tool`](https://docs.rs/swiftide/0.34/swiftide/attr.tool.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`AgentContext`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [`Command`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.Command.html) * [`CommandOutput`](https://docs.rs/swiftide/0.34/swiftide/traits/struct.CommandOutput.html) * [`CommandError`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.CommandError.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`ToolError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.ToolError.html) * [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) ## Next [Your first pipeline](/start/your-first-pipeline/) # Your first pipeline > Index a directory of Rust code into an embedded LanceDB table with local FastEmbed embeddings, then ask it one question and get an answer that cites files. This page indexes one directory of Rust code into an embedded LanceDB table and answers a question about it, naming the files the answer came from. You run the [ask-your-codebase](/showcases/ask-your-codebase/) showcase and read the parts that make it work. Everything except the LLM (large language model) runs on your machine. Tree-sitter chunks the code, FastEmbed embeds it on the CPU (central processing unit), and LanceDB stores the vectors in a local directory. ## Showcase run The showcase lives in `examples/ask-your-codebase/` in the swiftide repository. Clone it. To run exactly the code this site shows, check out the commit it is pinned to: ```sh git clone https://github.com/bosun-ai/swiftide.git cd swiftide git checkout e377c4a2e76fe2559629ae0efdae3c6e0fb4d829 # optional: the commit this site documents export OPENAI_API_KEY=sk-... cargo run -p ask-your-codebase -- index swiftide-agents cargo run -p ask-your-codebase -- ask --path swiftide-agents "How does an agent decide to stop?" ``` ## Dependencies To build the same pipelines in your own crate, add the showcase’s dependencies: ```sh cargo add swiftide@0.34 --no-default-features --features tree-sitter,fastembed,lancedb,redb,openai cargo add fastembed@5 --no-default-features cargo add tokio --features full cargo add anyhow async-trait ``` Each integration has its own [feature flag](/reference/feature-flags/). The direct `fastembed` dependency picks a model other than the default. Only the showcase’s own chunker needs `async-trait`. ## LLM client examples/ask-your-codebase/src/llm.rs ```rust impl LlmEndpoint { /// Builds a client for this endpoint. /// /// Rate limits and transient errors are retried with exponential backoff, which matters when /// indexing fires many prompts concurrently. After `retry_budget` the error is returned, so /// an unreachable or broken endpoint fails instead of hanging. pub fn client(&self, retry_budget: Duration) -> Result> { let openai = OpenAI::builder() .api_base(&self.base_url) .api_key(&self.api_key) .default_prompt_model(&self.model) .use_responses_api(matches!(self.api, OpenAiApi::Responses)) .build()?; let backoff = BackoffConfiguration { max_elapsed_time_sec: retry_budget.as_secs(), ..BackoffConfiguration::default() }; Ok(Arc::new(LanguageModelWithBackOff::new(openai, backoff))) } } ``` The client is swiftide’s `OpenAI` integration pointed at a base URL with `api_base` and `api_key`. Indexing sends many prompts at once, so `LanguageModelWithBackOff` wraps it and retries transient errors, such as rate limits, with exponential backoff. A wrong key is a permanent error and fails at once. `max_elapsed_time_sec` in `BackoffConfiguration` caps how long one prompt is [retried](/models/errors-backoff-and-usage/); the showcase allows 60 seconds while indexing and 30 while asking. ## Embedder and storage examples/ask-your-codebase/src/embedding.rs ```rust /// A code-aware embedding model that runs on the CPU and needs no API key. /// /// The model (~600MB) is downloaded on first use. pub fn local_model() -> Result { let model = TextEmbedding::try_new( InitOptions::new(EmbeddingModel::JinaEmbeddingsV2BaseCode).with_max_length(MAX_TOKENS), )?; FastEmbed::builder() .embedding_model(model) .batch_size(BATCH_SIZE) .build() } ``` The embedder runs `jina-embeddings-v2-base-code`, a model trained on code, with up to 2048 tokens per text. Its vectors have 768 dimensions, kept in `DIMENSIONS`. `BATCH_SIZE` (32) is the batch size FastEmbed uses inside one embedding call. Keep it equal to the batch size of the pipeline step that calls the embedder. examples/ask-your-codebase/src/codebase.rs ```rust /// Remembers which files were fully indexed, so unchanged files are skipped next time. pub fn node_cache(&self) -> Result { std::fs::create_dir_all(&self.data_dir)?; Redb::builder() .database_path(self.data_dir.join("cache.redb")) .table_name("indexed_files") .build() } /// Embedded vector store with one vector per chunk and the source path next to it. pub fn vector_store(&self) -> Result { let uri = self.lancedb_dir(); LanceDB::builder() .uri(uri.to_str().context("Index path is not valid UTF-8")?) .table_name("code_chunks") .vector_size(embedding::DIMENSIONS) .with_vector(EmbeddedField::Combined) .with_metadata(PATH_FIELD) .build() } ``` `Redb` is the node cache, a single embedded file. `LanceDB` is the storage: `uri` is a local directory, and `vector_size` must equal the embedder’s dimensions. `EmbeddedField::Combined` stores one vector per chunk, computed from the chunk and its metadata. `with_metadata(PATH_FIELD)` keeps the file path in its own column, so the answer can cite it. Both live in `/.ask-your-codebase/`, inside the indexed directory. ## Indexing pipeline examples/ask-your-codebase/src/index.rs ```rust let stats = indexing::Pipeline::from_loader( FileLoader::new(codebase.root()).with_extensions(&languages.extensions()), ) .filter(test_code::not_test_files(codebase.clone())) // Drop unit tests before the cache, which keys on path and content. Changing the content // after it would make the cache miss on every run. .then(test_code::without_unit_tests) .then(files_found.count()) // Skip files whose path and content are unchanged since the last successful run .filter_cached(codebase.node_cache()?) .then(files_changed.count()) .then(record_relative_path(codebase.clone())) // Split along syntax boundaries (functions, impls, ...) with tree-sitter .then_chunk(ChunkByLanguage::try_new(languages, CHUNK_SIZE)?) // Ask the LLM which questions each chunk answers; stored as metadata and embedded .then( MetadataQACode::from_client(llm) .prompt_template(QUESTIONS_PROMPT) .num_questions(QUESTIONS_PER_CHUNK) .concurrency(concurrent_requests) .build()?, ) // Embed chunk + metadata locally, in batches .then_in_batch(Embed::new(embedding::local_model()?).with_batch_size(embedding::BATCH_SIZE)) .then_store_with(codebase.vector_store()?) .run() .await?; ``` The indexing pipeline is a stream of nodes. Nothing runs until `run()` is awaited. `run` returns the pipeline statistics; the showcase reports `nodes_stored` as the chunk count. The marked lines are the core: * The loader, `FileLoader`, reads the files under the root with the given extensions. It skips hidden files and what `.gitignore` and `.ignore` files exclude. `.gitignore` only applies inside a git repository. Each file becomes one node. * `filter_cached` drops nodes the node cache has seen before. The cache key is a hash of the path and the content, so an edited file counts as new. * `then_chunk` runs the chunker. `ChunkByLanguage` is a small showcase type that picks a tree-sitter `ChunkCode` per file extension. It splits along syntax boundaries into chunks of 10 to 4096 bytes. * `then` runs a transformer on each chunk. `MetadataQACode` asks the LLM, with a custom prompt, for eight questions the chunk answers and stores them as metadata. `concurrency` caps the prompts in flight. * `then_in_batch` runs a batch transformer. `Embed` sends 32 chunks per call to FastEmbed. * `then_store_with` hands each chunk to the storage, here LanceDB, through the `Persist` trait. The other lines are showcase extras that drop tests, record the relative path and count for the report. Leave them out in your own crate, and replace the showcase helpers: * `FileLoader::new(codebase.root()).with_extensions(&languages.extensions())` becomes `FileLoader::new("./src").with_extensions(&["rs"])`. * `ChunkByLanguage::try_new(languages, CHUNK_SIZE)?` becomes `ChunkCode::try_for_language_and_chunk_size("rust", 10..4096)?`. * `codebase.node_cache()?` and `codebase.vector_store()?` are the builders from the storage excerpt. * `llm` is the client from the LLM client excerpt. Without `prompt_template`, `MetadataQACode` uses its default prompt. ## Query pipeline examples/ask-your-codebase/src/query.rs ```rust let search = SimilaritySingleEmbedding::default().with_top_k(top_k); let pipeline = query::Pipeline::from_search_strategy(search) // Rephrase the question into several, to widen what the search can match .then_transform_query( GenerateSubquestions::builder() .client(llm.clone()) .prompt_template(SUBQUESTIONS_PROMPT.into()) .build()?, ) // Embed with the same local model that embedded the chunks .then_transform_query(Embed::from_client(embedding::local_model()?)) .then_retrieve(codebase.vector_store()?) // Answer the original question from the retrieved code, each chunk under its file path .then_answer( answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(CHUNK_TEMPLATE) .build()?, ); let answered = pipeline.query(question).await?; ``` `GenerateSubquestions` rewrites the question into several related ones, to widen what the search can match. The query transformer `Embed` turns them into one vector with the same local model that embedded the chunks. The search strategy, `SimilaritySingleEmbedding`, runs one vector search with that embedding and returns the `top_k` closest chunks. Its default is 10; the showcase passes 20. `then_retrieve` uses the same `LanceDB` store as the retriever. The answer generator, `answers::Simple`, answers from the retrieved chunks; its document template renders each chunk under its file path. [Answering](/querying/answering/) covers other answer generators. `query` consumes the pipeline and returns the answered query. To ask more than once, call `query_mut` or `query_all`. ## Output This output is from the showcase’s `OUTPUT.md`. The chunk questions came from claude-haiku-5-5 and the answer from claude-sonnet-5-5, both through an OpenAI-compatible gateway. The embeddings came from FastEmbed on the CPU. The first run indexes everything: ```text Indexing Rust files in swiftide-agents ... Indexed 34 chunks from 14 new or changed files (14 files found, 0 unchanged) in 91.9s ``` A second run finds every file in the node cache and sends no prompts: ```text Indexing Rust files in swiftide-agents ... Indexed 0 chunks from 0 new or changed files (14 files found, 14 unchanged) in 0.6s ``` Asking “How does an agent decide to stop?” gives, trimmed: ```text # How an agent decides to stop An agent stops when something calls `Agent::stop` (or `stop_with_error`), which moves the state to `State::Stopped(StopReason)`. The main loop in `run_agent` checks that state after each step. `Agent::stop` is a no-op if the agent is already stopped (`src/agent.rs`). ## The stop mechanism - `Agent::stop(reason)` fires the `OnStop` hooks with the reason, then sets `self.state = State::Stopped(reason)`. It only takes effect at the end of the current loop iteration (`src/agent.rs`). - `Agent::stop_with_error(&AgentError)` fires `OnStop` with `StopReason::Error` and the error, then sets `Stopped(StopReason::Error)` (`src/agent.rs`). - You can inspect the result with `stop_reason()`, `is_stopped()` and `state()` (`src/agent.rs`, `src/state.rs`). ... Sources: - src/agent.rs - src/tools/control.rs - src/lib.rs - src/state.rs - src/errors.rs - src/task_agent.rs - src/hooks.rs - src/tasks.rs - src/default_context.rs ``` ## Variations ### OpenAI embeddings The LanceDB example embeds with OpenAI’s `text-embedding-3-small` instead, so its `vector_size` is 1536: ```rust let openai_client = integrations::openai::OpenAI::builder() .default_embed_model("text-embedding-3-small") .default_prompt_model("gpt-4o-mini") .build()?; let tempdir = TempDir::new().unwrap(); // Configure lancedb with a default vector size, a single embedding // and in addition to embedding the text metadata, also store it in a field let lancedb = LanceDB::builder() .uri(tempdir.child("lancedb").to_str().unwrap()) .vector_size(1536) .with_vector(EmbeddedField::Combined) .with_metadata(METADATA_QA_TEXT_NAME) .table_name("swiftide_test") .build() .unwrap(); ``` ### Qdrant with the default FastEmbed model `FastEmbed::builder()` without a model uses the default model, with 384 dimensions. This pipeline stores into Qdrant. It needs the `qdrant` feature flag and a Qdrant server at `QDRANT_URL`, which defaults to `http://localhost:6334`. See [Qdrant](/integrations/qdrant/): ```rust indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["rs"])) .then_in_batch(Embed::new(FastEmbed::builder().batch_size(10).build()?)) .then_store_with( Qdrant::try_from_url(qdrant_url)? .batch_size(50) .vector_size(384) .collection_name("swiftide-examples-fastembed".to_string()) .build()?, ) .run() .await?; ``` For hybrid search and reranking, see the [docs-qa](/showcases/docs-qa/) showcase. ## Gotchas * Use the same embedding model for indexing and querying, and set `vector_size` to its dimensions. After you change the model, delete `/.ask-your-codebase/` in the indexed directory and index again. * The first run downloads `jina-embeddings-v2-base-code`, about 600MB, into `.fastembed_cache/` in the current directory. * When a file changes, its new chunks are added, but the chunks of the old version stay in LanceDB. * Files enter the node cache only after `run()` finishes without an error. One failed chunk ends the run, and every file of that run is prompted and embedded again next time. The same holds for a run you interrupt. * A file that produces no chunks is never cached, so it is loaded again on every run. ## Reference * [`indexing::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`NodeCache`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.NodeCache.html) * [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`BackoffConfiguration`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.BackoffConfiguration.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkCode.html) * [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQACode.html) * [`transformers::Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`Redb`](https://docs.rs/swiftide/0.34/swiftide/integrations/redb/struct.Redb.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`query_transformers::Embed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.Embed.html) * [`GenerateSubquestions`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.GenerateSubquestions.html) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) ## Next [Your first task](/start/your-first-task/) # Your first task > Build a typed task with two nodes and one transition, run it to completion and print a one-line report of crate names. You end this page with a `Task` that turns a list of crate names into a one-line report. It has two task nodes, one transition between them and a finish. The [workspace-architect](/showcases/workspace-architect/) showcase starts the same way, with a `discover` node, before it fans out to one agent per crate. ## Dependencies The task needs no model, so the `swiftide-tasks` feature flag is enough. `swiftide-agents` also turns `swiftide-tasks` on. ```sh cargo add swiftide@0.34 --features swiftide-tasks cargo add tokio --features macros,rt-multi-thread cargo add anyhow ``` ## Program `main` builds the task and runs it: src/main.rs ```rust use swiftide::tasks::{NodeError, Task, TaskRunOutcome}; #[tokio::main] pub async fn main() -> anyhow::Result<()> { // A task from a manifest listing (one crate per line) to a one-line report. let mut task = Task::::new(); let discover = task.register_node_fn(|listing: &String| { let crates: Vec = listing.lines().map(str::trim).map(String::from).collect(); Ok::<_, NodeError>(crates) }); let report = task.register_node_fn(|crates: &Vec| { Ok::<_, NodeError>(format!("{} crates: {}", crates.len(), crates.join(", "))) }); task.starts_with(discover); task.register_transition(discover, move |crates| report.transitions_with(crates))?; task.register_transition(report, task.transitions_to_finish())?; let listing = "swiftide-core\nswiftide-tasks\nswiftide-agents".to_string(); match task.run(listing).await? { TaskRunOutcome::Completed(line) => println!("{line}"), TaskRunOutcome::Paused => println!("paused"), } Ok(()) } ``` ## Task nodes `Task::::new()` creates an empty task. The first type parameter is the input you pass to `run`. The second is the output it returns. `register_node_fn` turns a synchronous closure into a task node. The closure takes a reference to its input and returns a `Result`. The error type must implement `std::error::Error + Send + Sync + 'static`, so `anyhow::Error` does not fit. The closures never fail, so `Ok::<_, NodeError>(..)` only names a type the compiler can infer. The method returns a typed `NodeId`, which you use to wire transitions. `discover` splits the listing into a `Vec`. `report` takes that vector and formats a line. Every input and output type must be `Clone + Debug + Send + Sync + 'static`. The `NodeArg` trait covers exactly that bound, so plain owned types work. ## Transitions `starts_with(discover)` marks the node that receives the task input. `register_transition` connects a node to what runs after it. The closure receives the output of `discover` and returns `report.transitions_with(crates)`. The compiler checks that the input type of `report` matches the output type of `discover`. A mismatch is a compile error, not a runtime surprise. `task.transitions_to_finish()` ends the task with the output of `report`. The finishing node’s output type must equal the task’s output type. Each node gets exactly one transition. A second `register_transition` for the same node returns an error. A node without a transition makes `run` fail with `TaskError::MissingTransition`, and a task without a start node fails with `TaskError::NoSteps`. ## Run outcome `task.run(input)` returns `Result, TaskError>`. `TaskRunOutcome::Completed` carries the output. `TaskRunOutcome::Paused` means a node asked the task to pause, which this graph never does. `cargo run` prints: ```text 3 crates: swiftide-core, swiftide-tasks, swiftide-agents ``` A node error stops the run and comes back as `TaskError::NodeError`. Its message gives the numeric id of the node that failed, as in `Node error in node 0: ...`. The id is the order in which you registered the node, not the name of the variable that holds it. To run the same graph again with the last input, call `reset` and then `resume`. ## Fan-out A transition can also fan out. It schedules one branch per input on the same node, and a join node collects the outputs. The [workspace-architect showcase](/showcases/workspace-architect/) uses this to run one agent per crate. [Fan-out and join](/tasks/fan-out-and-join/) covers the API. ## Common mistakes * Returning `anyhow::Result` from a `register_node_fn` closure does not compile. Use an error type that implements `std::error::Error`, such as your own `thiserror` enum. * `run` on a task that is still active or paused returns `TaskError::TaskActive`. Continue a paused task with `resume`. * A human approval step is `Transition::pause()` followed by `task.resume()`. See [Pause and resume](/tasks/pause-and-resume/). * For nodes that call a model, see [Nodes and transitions](/tasks/nodes-and-transitions/) and [Agents as nodes](/tasks/agents-as-nodes/). An agent node also needs the `swiftide-agents` feature flag and a provider feature flag such as `openai`. ## Reference * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html) * [`TaskRunOutcome`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskRunOutcome.html) * [`NodeId`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeId.html) * [`NodeError`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeError.html) * [`TaskError`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskError.html) * [`Transition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html) ## Next [Using Swiftide with coding agents](/start/for-coding-agents/) # Context and message history > Store an agent's message history in memory or Redis, resume it in a new process, and rewrite it while the agent runs. An agent’s message history can outlive the process and be rewritten while the agent runs. The `agents_resume.rs` example saves the history of one agent as JSON (JavaScript Object Notation) and loads it into a second one: ```rust // Let's store the messages in a database, retrieve them back, and start a new agent let stored_history = serde_json::to_string(&first_agent.history().await?)?; let retrieved_history: Vec<_> = serde_json::from_str(&stored_history)?; let restored_context = DefaultContext::default() .with_existing_messages(retrieved_history) .await? .to_owned(); ``` ```rust let mut second_agent = agents::Agent::builder() .llm(&openai) .context(restored_context) // We'll use the one from the first agent, alternatively we could also pop it from the // previous history and add a new one here .no_system_prompt() .build()?; second_agent.query("What did you say?").await?; ``` `Agent::history()` returns every `ChatMessage` the context holds. `ChatMessage` implements `Serialize` and `Deserialize`, so `serde_json` turns the history into a string you can store anywhere. `DefaultContext::with_existing_messages` loads the messages back into a fresh context. The second agent gets that context through the builder’s `context` method and continues the conversation with `query`. The next completion sends the whole restored history to the model. The first agent sets no system prompt, so `no_system_prompt()` changes nothing in this example. [Prompt removal](/agents/system-prompts/#prompt-removal) explains when it matters. `run` without a new message only continues when the history ends in a user message, a tool output or a pending tool call. When it ends in a plain assistant answer, as it does here, `run` stops at once with `StopReason::NoNewMessages`. ## Default context The context stores the message history, decides when the next completion runs and what it contains, holds the tool executor and records feedback for tools that wait on a person. Every agent built without `.context(..)` gets a `DefaultContext`: * the message history is an in-memory `Mutex>`; * tools run commands through a `LocalExecutor` in the current directory; * the agent stops when the last message is an assistant answer and no tool feedback is pending. `DefaultContext::from_executor` keeps those defaults and swaps the tool executor. It takes a value that converts into `Arc`. [Tool executors](/agents/executors/) covers the conversion and the tool executors themselves. `with_stop_on_assistant(false)` turns off the last rule. A text answer then no longer ends the run, and the agent asks the model again. Only a tool that stops the agent, the builder’s `limit`, or a model reply with no content and no tool calls ends the loop. An agent has no limit by default, so set one ([Stop reasons](/agents/stopping-and-structured-results/#stop-reasons)). A clone of a `DefaultContext` shares its history and counters. ## Message history Where the messages live is the job of the `MessageHistory` trait. It has three required methods: * `history` returns all messages in order; * `push_owned` appends one message; * `overwrite` replaces all messages. The trait requires `Send + Sync + Debug` and is declared with `#[async_trait]`, so annotate your impl the same way. `push`, `extend` and `extend_owned` have default implementations on top of `push_owned`. Implement the trait for your database and pass the value to `DefaultContext::with_message_history`. The backend only stores the list. Each completion sends the full history, cut at the latest summary, and the context’s counters only decide whether a new completion runs. ### Custom backend This backend keeps one conversation in a JSON file. It uses the `async-trait` crate for the `#[async_trait]` attribute: ```rust /// Keeps the message history of one conversation in a JSON file. #[derive(Debug)] struct JsonFileHistory { path: PathBuf, // Serializes read-modify-write cycles on the file. lock: Mutex<()>, } impl JsonFileHistory { async fn read(&self) -> anyhow::Result> { match tokio::fs::read_to_string(&self.path).await { Ok(json) => Ok(serde_json::from_str(&json)?), Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(Vec::new()), Err(err) => Err(err.into()), } } async fn write(&self, messages: &[ChatMessage]) -> anyhow::Result<()> { tokio::fs::write(&self.path, serde_json::to_vec(messages)?).await?; Ok(()) } } #[async_trait] impl MessageHistory for JsonFileHistory { async fn history(&self) -> anyhow::Result> { let _guard = self.lock.lock().await; self.read().await } async fn push_owned(&self, item: ChatMessage) -> anyhow::Result<()> { let _guard = self.lock.lock().await; let mut messages = self.read().await?; messages.push(item); self.write(&messages).await } async fn overwrite(&self, items: Vec) -> anyhow::Result<()> { let _guard = self.lock.lock().await; self.write(&items).await } } ``` ```rust let mut context = DefaultContext::default(); context.with_message_history(JsonFileHistory { path: "session-42.json".into(), lock: Mutex::new(()), }); let mut agent = Agent::builder().llm(&openai).context(context).build()?; ``` Every method takes `&self`, so the backend guards its own state; here a `tokio::sync::Mutex` keeps two writes from interleaving. `history` returns an empty list for a file that does not exist yet. `with_message_history` takes the value by ownership and stores it behind an `Arc`. ## Redis storage The `redis` feature flag adds a `Redis` integration that implements `MessageHistory`: ```sh cargo add swiftide@0.34 --features swiftide-agents,openai,redis ``` ```rust let redis = Redis::try_from_url("redis://localhost:6379", "session-42")?; let mut context = DefaultContext::default(); context.with_message_history(redis); let mut agent = Agent::builder() .llm(&openai) .context(context) .system_prompt("You help maintain the swiftide repository.") .build()?; agent.query("Which crates changed this week?").await?; ``` The messages go into a Redis list under `:message_history`, here `session-42:message_history`, one JSON string per message. Use one prefix per conversation. A new process that builds a context with the same prefix sees the whole conversation again. Build that agent without `.system_prompt(..)`, or with `no_system_prompt()`, as in the first example. Otherwise its first run pushes another system message onto the list. The connection opens on first use. If Redis is unreachable, the agent returns `AgentError::MessageHistoryError`. ## Approvals across restarts Approvals are not part of the message history. They live in the context’s memory. When the agent stops with `StopReason::FeedbackRequired`, store the `ToolCall` it carries; the person’s `ToolFeedback` comes later. Store each decision as a `(ToolCall, ToolFeedback)` pair, since `serde_json` cannot use a struct as a map key. Both types implement `Serialize` and `Deserialize`. Collect the pairs into a `HashMap` for `DefaultContext::with_tool_feedback` in the new process. [Human in the loop](/agents/human-in-the-loop/) covers approvals. ```rust // First process: the agent waits for approval; store the person's decision with its call. if let Some(StopReason::FeedbackRequired { tool_call, .. }) = agent.stop_reason() { let decisions = vec![(tool_call.clone(), ToolFeedback::approved())]; std::fs::write("approvals.json", serde_json::to_string(&decisions)?)?; } // Second process: load the decisions next to the restored message history, then resume. let decisions: Vec<(ToolCall, ToolFeedback)> = serde_json::from_str(&std::fs::read_to_string("approvals.json")?)?; let mut context = DefaultContext::default(); context.with_message_history(Redis::try_from_url("redis://localhost:6379", "session-42")?); context.with_tool_feedback(decisions.into_iter().collect::>()); let mut resumed = Agent::builder() .llm(&openai) .context(context) .no_system_prompt() .build()?; resumed.run().await?; ``` `with_tool_feedback` accepts anything that converts into a `HashMap` and adds the entries to the feedback the context already holds. The restored history still ends in the pending tool call, so `run` calls the tool again, and the tool now finds its approval. ## History rewrites Since 0.33, `AgentContext::replace_history` swaps the whole history while the agent runs. It takes `&self`, so any hook can call it through `agent.context()`. This `after_each` hook shortens old tool outputs once the history passes `MAX_MESSAGES`: ```rust const MAX_MESSAGES: usize = 40; const KEEP_RECENT: usize = 10; let mut agent = Agent::builder() .llm(&openai) .after_each(|agent: &mut Agent| { Box::pin(async move { let history = agent.history().await?; if history.len() <= MAX_MESSAGES { return Ok(()); } let keep_from = history.len() - KEEP_RECENT; let trimmed = history .into_iter() .enumerate() .map(|(index, message)| match message { ChatMessage::ToolOutput(call, ToolOutput::Text(_)) if index < keep_from => { ChatMessage::ToolOutput(call, ToolOutput::text("(output removed)")) } other => other, }) .collect(); agent.context().replace_history(trimmed).await?; Ok(()) }) }) .build()?; ``` The hook only replaces the text of older tool outputs. Each tool call keeps its output, which the providers require. `DefaultContext` writes the new list through `overwrite` and resets its completion counters. The next completion then runs even if no new message was added since the last one. ## Summaries A `ChatMessage::Summary` shortens what the model sees without rewriting storage. When the history holds one, `DefaultContext` sends the system messages plus everything from the latest summary on. Add one with `ChatMessage::new_summary` through `agent.context().add_message(..)`. OpenAI-compatible providers send a summary as an assistant message. `ContextCompression` in agents-contrib writes these summaries for you; see [Contrib](/agents/contrib/). ## Custom contexts For behavior the message history cannot express, implement `AgentContext` yourself. Wrap a `DefaultContext` and forward the calls you do not change. The [upgrading guide](/reference/upgrading/) has a context that counts added messages: ```rust /// Counts the messages an agent adds and delegates everything else. #[derive(Clone)] pub struct CountingContext { inner: DefaultContext, added: Arc, } #[async_trait] impl AgentContext for CountingContext { async fn next_completion(&self) -> anyhow::Result>> { self.inner.next_completion().await } async fn current_new_messages(&self) -> anyhow::Result> { self.inner.current_new_messages().await } async fn add_messages(&self, items: Vec) -> anyhow::Result<()> { self.added .fetch_add(items.len(), std::sync::atomic::Ordering::Relaxed); self.inner.add_messages(items).await } async fn add_message(&self, item: ChatMessage) -> anyhow::Result<()> { self.added .fetch_add(1, std::sync::atomic::Ordering::Relaxed); self.inner.add_message(item).await } #[allow(deprecated)] async fn exec_cmd(&self, cmd: &Command) -> Result { self.inner.exec_cmd(cmd).await } fn executor(&self) -> &Arc { self.inner.executor() } async fn history(&self) -> anyhow::Result> { self.inner.history().await } async fn replace_history(&self, items: Vec) -> anyhow::Result<()> { self.inner.replace_history(items).await } async fn redrive(&self) -> anyhow::Result<()> { self.inner.redrive().await } async fn has_received_feedback(&self, tool_call: &ToolCall) -> Option { self.inner.has_received_feedback(tool_call).await } async fn feedback_received( &self, tool_call: &ToolCall, feedback: &ToolFeedback, ) -> anyhow::Result<()> { self.inner.feedback_received(tool_call, feedback).await } } ``` ## Gotchas * `with_existing_messages` calls `overwrite` on the backend. With Redis it deletes the list under the key and writes the given messages. * `with_existing_messages` and `with_message_history` return `&mut DefaultContext`. Use a `let mut` binding, or end the chain with `.to_owned()` as the example does. * A builder-made `Redis` uses the key `message_history` unless you set `message_history_key(..)` on the builder or call `with_message_history_key` afterwards. Agents that share that key also share one history. * Messages added through `agent.context()` do not trigger `on_new_message` hooks. Only `Agent::add_message` runs them. ## Reference * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`DefaultContext::with_existing_messages`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html#method.with_existing_messages) * [`DefaultContext::with_message_history`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html#method.with_message_history) * [`DefaultContext::with_tool_feedback`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html#method.with_tool_feedback) * [`AgentContext`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html) * [`AgentContext::replace_history`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html#tymethod.replace_history) * [`MessageHistory`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.MessageHistory.html) * [`Redis`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.Redis.html) * [`Redis::try_from_url`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.Redis.html#method.try_from_url) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`ChatMessage::new_summary`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html#method.new_summary) * [`ToolFeedback`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.ToolFeedback.html) * [`AgentBuilder::context`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.context) * [`AgentBuilder::no_system_prompt`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.no_system_prompt) * [`Agent::history`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html#method.history) ## Next [Tool executors](/agents/executors/) # Contrib tools and hooks > Give an agent file tools (read_file, edit, apply_patch), an update_plan tool, hooks that truncate tool output and compress the conversation, and an executor that starts on first use, from swiftide-agents-contrib. With the components on this page, your agent reads and edits files, keeps a plan, and keeps its context small on long runs. They come from swiftide-agents-contrib. ```rust let openai = OpenAI::builder().default_prompt_model("gpt-5.4").build()?; let executor: Arc = Arc::new(LocalExecutor::new("/srv/checkout")); let mut agent = Agent::builder() .llm(&openai) .context(DefaultContext::from_executor(executor)) .system_prompt("You fix failing tests. Keep a plan with update_plan.") .add_tool(ReadFile::default().with_line_numbers()) .add_tool(ExactReplace) .add_tool(ApplyPatch) .add_tool(UpdatePlan::default()) .limit(50) .build()?; agent .query("The test `parses_empty_input` in src/parser.rs fails. Fix it.") .await?; ``` `limit(50)` caps the run at 50 completions, because a fixing loop can run long. The four contrib tools sit next to the `stop` tool that every agent gets. Every file operation runs as a command through the tool executor of the context, here a `LocalExecutor` rooted at the checkout. ## Feature flags Nothing in the crate is enabled by default. Pick one feature flag per capability: | Feature flag | Adds | | ------------------------------ | ---------------------------------------------------- | | `agents-contrib-file-tools` | `ReadFile`, `ExactReplace`, `ApplyPatch` | | `agents-contrib-planning` | `UpdatePlan`, `PlanItem`, `PlanStatus`, `PlanMemory` | | `agents-contrib-context` | `ToolSummarizer`, `ContextCompression` | | `agents-contrib-lazy-executor` | `LazyExecutor`, `ToolExecutorExt` | Each one turns on `agents-contrib`, which turns on `swiftide-agents`. The plain `agents-contrib` flag only exposes the crate as `swiftide::agents_contrib`: ```sh cargo add swiftide@0.34 --features openai,agents-contrib-file-tools,agents-contrib-planning cargo add tokio --features full cargo add anyhow ``` A feature flag never registers a tool or a hook. You add each component to the builder yourself. [Feature flags](/reference/feature-flags/) lists every flag. ## File tools The model sees these tool names: | Type | Tool name | Arguments | | ----------------------------------------- | ----------------------------- | --------------------------------------------------------------- | | `ReadFile::default()` | `read_file` | `file_name`, optional `offset` and `limit` | | `ReadFile::default().with_line_numbers()` | `read_file_with_line_numbers` | same, output prefixed with `N\|` line numbers | | `ExactReplace` | `edit` | `file_path`, `old_string`, `new_string`, optional `replace_all` | | `ApplyPatch` | `apply_patch` | `patch` | `read_file` returns up to 2,000 lines by default. It cuts single lines at 2,000 characters and refuses a window over 100,000 characters. When it stops early, the output tells the model which offset to read next. `edit` replaces a match only when it is unique, unless the model sets `replace_all`. `apply_patch` takes a patch in this format: ```text *** Begin Patch *** Add File: hello.txt +Hello *** Update File: src/main.rs @@ fn greet() - println!("Hi"); + println!("Hello"); *** End Patch ``` `*** Move to: new/path` after an update header moves a file. `*** Delete File: path` deletes one. Lines starting with a space are context, `-` removes a line and `+` adds one. The file tools need a tool executor with a POSIX (Portable Operating System Interface) shell and the `cat`, `mkdir` and `rm` commands. ## Plans `UpdatePlan` registers `update_plan`. The model sends `items`, each with a `step` and a `status` of `pending`, `in_progress` or `completed`, plus an `explanation` string. Both arguments are required. An empty `items` list returns the current plan without changing it. The tool answers with a `ToolOutput::fail` when a step is empty or spans several lines, or when more than one step is in progress. The stored plan stays as it was. Clones of one `UpdatePlan` share the plan, while two `UpdatePlan::default()` values each have their own. The plan lives in memory and is not persisted. ## Context management Two hooks keep the message history short: ```rust let small_model = OpenAI::builder() .default_prompt_model("gpt-5-mini") .build()?; let summarizer = ToolSummarizer::builder() .llm(Arc::new(small_model.clone()) as Arc) .tools_to_summarize(vec!["run_tests"]) .git_start_sha("HEAD") .build()?; let compression = ContextCompression::builder() .llm(Arc::new(small_model) as Arc) .git_start_sha("HEAD") .git_diff_dir(".") .context_limit(200_000usize) .build()?; builder.after_tool(summarizer.summarize_hook()); compression.register_hooks(builder); ``` Call both on the same `AgentBuilder` before `build()`. `tools_to_summarize` only applies to tools the agent has registered. A listed tool the agent lacks is skipped with a warning. `run_tests` here stands for a tool you define yourself. `ToolSummarizer::summarize_hook` returns an `after_tool` hook. It truncates any tool output over an estimated 10,000 tokens to its first 40 and last 160 lines. It then cuts the result to a character budget of four characters per token, 40,000 by default, so a long line can still end mid-line. Finally, it summarizes the output of the tools named in `tools_to_summarize` with the model you pass to `llm`. Set `summarize_all_at_token_cutoff` to summarize any large output. By default, `skip_summarize_all` exempts the file-read tools from truncation and from `summarize_all_at_token_cutoff`. For the tools in `tools_needing_diff` (default `run_tests` and `run_coverage`), the summary prompt includes `git diff` against `git_start_sha`. `ContextCompression::register_hooks` adds an `after_completion` and an `after_each` hook. They check three triggers in this order: 1. A turn that ends with a file read postpones any summary until the history reaches 90% of `context_limit`. 2. The sixth turn after the last summary triggers a new summary (`num_completions_for_summary`). 3. A history estimated above 70% of `context_limit` triggers one, unless the agent has stopped. The model you pass to `llm` writes the summary from the history, a Git diff and the plan progress. The hook stores the summary as a `ChatMessage::Summary`. A `DefaultContext` then sends only the system messages and everything from the latest summary on ([Summaries](/agents/context-and-history/#summaries)). When the summary interrupts a tool turn, the hook also appends a user message `Continue` to the message history. After the first summary, the hook adds a `read_original_instructions` tool so the model can recover the first user message. It adds the tool only when the history holds a user message. `ContextCompression`, and `ToolSummarizer` for the tools in `tools_needing_diff`, run `git diff` through the agent’s tool executor. Point that tool executor at a Git repository. ## Lazy executor `LazyExecutor` defers expensive setup, such as cloning a repository or starting a container, until the agent first needs it: ```rust let lazy = LazyExecutor::new(|| async { // Clone a repository, start a container, ... before the first command runs. Ok(Arc::new(LocalExecutor::new("/srv/checkout")) as Arc) }); assert!(!lazy.is_started()); let executor: Arc = Arc::new(lazy); let agent = Agent::builder() .llm(openai) .context(DefaultContext::from_executor(executor)) .add_tool(ReadFile::default()) .build()?; ``` `LazyExecutor::new` takes a closure whose future resolves to an `anyhow::Result>`. The first command, file stream or download calls it. Clones share the started executor. A failed start is retried on the next use. `is_started` returns true once the closure has succeeded. `ToolExecutorExt` adds `is_started`, `cleanup` and `download` to a tool executor, so bring the trait into scope to call them. It is implemented for `LocalExecutor`, for `LazyExecutor` itself and for a `ScopedExecutor` whose inner executor implements `ToolExecutorExt` and `Clone`. `cleanup` before the start does nothing. `download` runs `dd`, `base64` and `sha256sum` in the tool executor. ## Gotchas * `LocalExecutor` does not isolate the file system. `read_file` and `edit` pass absolute paths to the tool executor unchanged. `apply_patch` only takes relative paths without `..` components, but it follows symlinks like the other tools. Use an isolated executor, such as the separate swiftide-docker-executor crate, for repositories you do not trust. * Patches are not transactions. `apply_patch` runs its operations in order and can partly succeed, so the model has to read the result report. * Tools are unique by name. If you already have a tool called `read_file` or `edit`, `add_tool` replaces it with the contrib tool, or the other way around. * The default `context_limit` is 400,000,000. Set one that matches your model, or compression only triggers on the turn count. * `ContextCompression` keeps its turn counter behind an `Arc`, shared with clones. Build a fresh one for every independent agent. * Summaries lose detail. Keep facts the model must not forget in the system prompt. * Debug logs contain the summary prompts and summaries, including file contents. ## Reference The links point at the source on GitHub. * [swiftide-agents-contrib README](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/README.md) * [`ReadFile`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/tools/read_file.rs), [`ExactReplace`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/tools/exact_replace.rs), [`ApplyPatch`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/tools/apply_patch/mod.rs) * [`UpdatePlan`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/tools/update_plan.rs) * [`ToolSummarizer`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/hooks/tool_summarizer.rs), [`ContextCompression`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/hooks/context_compression.rs) * [`LazyExecutor`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/executors/lazy.rs), [`ToolExecutorExt`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/executors/traits.rs) * [contrib examples](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/examples) * [`AgentBuilder::add_tool`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.add_tool), [`AgentBuilder::after_tool`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.after_tool) * [`DefaultContext::from_executor`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html#method.from_executor) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) * [Feature flags](/reference/feature-flags/) ## Next [Testing agents](/agents/testing/) # Tool executors > Run agent tool commands through a ToolExecutor: a LocalExecutor with a working directory, timeout and environment, scoped executors, streamed output, and model arguments validated before they reach a shell. Your tools run shell commands and file operations through a tool executor instead of spawning processes themselves. You end up with one that works in a chosen directory, stops slow commands, streams their output, and only receives arguments your code has checked. The local executor needs the `swiftide-agents` feature flag: ```sh cargo add swiftide@0.34 --features swiftide-agents ``` ## Local executor ```rust let executor = LocalExecutor::builder() .workdir("/srv/checkout") .default_timeout(Some(Duration::from_secs(30))) .env_remove(vec!["OPENAI_API_KEY".to_owned(), "GITHUB_TOKEN".to_owned()]) .envs(HashMap::from([( "CARGO_TERM_COLOR".to_owned(), "never".to_owned(), )])) .build()?; ``` The builder sets where and how commands run: * `workdir` sets the directory commands start in. It defaults to `.`. `LocalExecutor::new` only sets this field. * `default_timeout` takes an `Option` for commands without a timeout of their own. * `env_remove` drops variables, here secrets that the model’s commands do not need. * `envs` sets variables. They apply after `env_clear` and `env_remove`. * `env_clear(true)` (not shown) starts every command with an empty environment. That also drops `PATH` and `HOME`, so pass the ones your commands need back through `envs`. The agent gets the executor through its context. [Context and message history](/agents/context-and-history/) shows how `DefaultContext::from_executor` swaps it in. ## Tool executor trait `ToolExecutor` has three methods: * `exec_cmd` runs a `Command` and returns its `CommandOutput` or a `CommandError`. * `exec_cmd_streaming` also hands each piece of output to a closure. The default implementation reports all output once the command has finished. * `stream_files` returns the files below a path as an `IndexingStream` for an indexing pipeline; see [Loading](/indexing/loading/). A tool reaches the executor with `context.executor()`. `AgentContext::exec_cmd` is deprecated. ### Custom executor You write your own tool executor to run commands somewhere else, such as a remote machine, or to put a policy in front of another executor. This one wraps any executor and refuses file writes: ```rust /// Runs commands through another executor, but refuses to write files. #[derive(Clone)] pub struct ReadOnly { inner: E, } impl ReadOnly { fn check(cmd: &Command) -> Result<(), CommandError> { if matches!(cmd, Command::WriteFile { .. }) { return Err(CommandError::ExecutorError(anyhow::anyhow!( "this executor is read-only" ))); } Ok(()) } } #[async_trait] impl ToolExecutor for ReadOnly { async fn exec_cmd(&self, cmd: &Command) -> Result { Self::check(cmd)?; self.inner.exec_cmd(cmd).await } async fn exec_cmd_streaming( &self, cmd: &Command, output: &mut CommandOutputSink<'_>, ) -> Result { Self::check(cmd)?; self.inner.exec_cmd_streaming(cmd, output).await } async fn stream_files( &self, path: &Path, extensions: Option>, ) -> anyhow::Result> { self.inner.stream_files(path, extensions).await } } ``` The trait uses `#[async_trait]`, so add the `async-trait` crate. The trait extends `DynClone`, so your type must implement `Clone`. `exec_cmd` and `stream_files` are required. `ReadOnly` forwards `exec_cmd_streaming` as well; without it, the default would run `exec_cmd` and report the output only at the end. `Command` is non-exhaustive, so match it with a wildcard arm or `matches!`. Return `CommandError::ExecutorError` when the executor refuses or fails, and `NonZeroExit` when the command ran and failed. An executor that cannot stream can return `IndexingStream::from(anyhow::Error)`, so a loader reports the error instead of panicking. Give the agent your executor the same way as a local one: ```rust let executor: Arc = Arc::new(ReadOnly { inner: LocalExecutor::new("/srv/checkout"), }); let agent = Agent::builder() .llm(&openai) .context(DefaultContext::from_executor(executor)) .build()?; ``` `ReadOnly` only blocks `Command::WriteFile`. A shell string such as `echo hi > notes.md` still writes, so a real policy also has to check or forbid `Command::Shell`. ## Output streaming ```rust let (chunks, mut received) = tokio::sync::mpsc::unbounded_channel(); let printer = tokio::spawn(async move { while let Some(chunk) = received.recv().await { match chunk { CommandOutputChunk::Stdout(bytes) => print!("{}", String::from_utf8_lossy(&bytes)), CommandOutputChunk::Stderr(bytes) => eprint!("{}", String::from_utf8_lossy(&bytes)), } } }); let cmd = Command::shell("cargo test").with_timeout(Duration::from_secs(600)); let result = executor .exec_cmd_streaming(&cmd, &mut |chunk: &CommandOutputChunk| { // The sink runs inside the executor's read loop: hand the chunk off, never block. let _ = chunks.send(chunk.clone()); }) .await; drop(chunks); printer.await?; let output = result?; println!("{} bytes of output", output.len()); ``` `exec_cmd_streaming` passes each chunk to a sink while the command runs. The sink is a `CommandOutputSink`: a `FnMut(&CommandOutputChunk) + Send` closure. The executor calls the sink from its read loop, so hand each chunk off instead of doing work in it. The local executor streams shell commands and reports file commands once they finish. The return value matches `exec_cmd`. ## Commands You build a `Command` with `Command::shell`, `Command::read_file` or `Command::write_file`, as in the streaming example above and the tool further down. `Command::shell` runs a string with `sh -c`. The file commands skip the shell, so a path never needs quoting. Two methods set options on one command: * `with_current_dir` sets the working directory of the command. * `with_timeout` sets the timeout of the command. It wins over `default_timeout`, also when it is longer. How a relative directory or path resolves is up to the executor. The local executor joins a relative `current_dir` onto its `workdir` and uses an absolute one as it is. File paths resolve against the result. Its `write_file` creates missing parent directories and replaces the file’s content. ## Command output ```rust let cmd = Command::shell("cargo test").with_current_dir("crates/core"); match context.executor().exec_cmd(&cmd).await { Ok(output) => { let stdout: Vec = output.stdout().flat_map(|b| b.iter().copied()).collect(); println!("{}", String::from_utf8_lossy(&stdout)); } Err(CommandError::NonZeroExit(output)) => { for chunk in output.stderr() { eprintln!("{}", String::from_utf8_lossy(chunk)); } } Err(other) => return Err(other.into()), } ``` `CommandOutput` keeps stdout and stderr apart. It holds `CommandOutputChunk::Stdout` and `CommandOutputChunk::Stderr` chunks in the order the executor read them. `stdout()` and `stderr()` iterate over one stream; `to_string_lossy()` and `Display` join both. A `CommandOutput` converts into a `ToolOutput` with the joined text. A failed command returns an error that still carries its output: * `CommandError::NonZeroExit` holds the output of a command that exited with a non-zero code. An input/output (I/O) error, such as reading a missing file, also becomes `NonZeroExit`, with the error message as output. * `CommandError::TimedOut` holds the timeout and the output read so far. For a shell command, the local executor kills the process (its process group on Unix). A file command that times out returns empty output. * `CommandError::ExecutorError` means the executor itself failed. A tool that returns such an error with `?` sends its text to the model and counts toward the tool retry limit. See [Tool errors and retries](/agents/tools/#tool-errors-and-retries). Match `NonZeroExit` yourself when failing is a normal answer, like a failing test. ## Shebang scripts ```rust let script = "#!/usr/bin/env python3\nimport sys\nprint(sys.version_info.major)"; let output = executor.exec_cmd(&Command::shell(script)).await?; println!("python {}", output.to_string_lossy().trim()); ``` When a shell command starts with `#!`, the local executor starts the interpreter named on that line and writes the rest of the string to its stdin. Everything after the interpreter on the first line is passed as one argument, so `#!/usr/bin/env python3` runs `/usr/bin/env` with `python3`. A model can send a multi-line script without escaping it for `sh -c`. ## Scoped executors ```rust let checkout = LocalExecutor::new("/srv/checkout"); // Borrowed: one command in /srv/checkout/docs. let docs = checkout.scoped("docs"); let listing = docs.exec_cmd(&Command::shell("ls")).await?; println!("{listing}"); // Owned: commands without an absolute directory or path run in /srv/checkout/crates/core. let executor: Arc = Arc::new(checkout.scoped_owned("crates/core")); let agent = Agent::builder() .llm(&openai) .context(DefaultContext::from_executor(executor)) .build()?; ``` `ExecutorExt` adds `scoped`, which borrows the executor, and `scoped_owned`, which takes it so it can live as long as an agent. Both return a `ScopedExecutor`: * A command without a working directory runs in the scope. * A relative `current_dir` joins onto the scope. The local executor then joins the result onto its `workdir`. * An absolute `current_dir` is left alone, and so is an absolute file path. A scope does not keep a tool inside it. * `stream_files` joins a relative path onto the scope. The local executor does not add its `workdir` to that path; see the gotchas below. ## Argument validation A working directory is not a sandbox: it only chooses where a command starts. An absolute path, a `..` or a `;` in a shell string still reaches the rest of the machine. Parse model arguments into a type that cannot hold a bad value first. The release-notes-agent showcase validates every argument with `Revision` and `RepoPath` in [`src/git.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/git.rs). The same pattern for your own tool: ```rust /// A path inside the executor's working directory: relative and without `..`. #[derive(Debug, Clone)] pub struct WorkspacePath(PathBuf); impl FromStr for WorkspacePath { type Err = anyhow::Error; fn from_str(raw: &str) -> anyhow::Result { let path = Path::new(raw.trim()); ensure!(!path.as_os_str().is_empty(), "path is empty"); for component in path.components() { match component { Component::Normal(_) | Component::CurDir => {} Component::ParentDir => bail!("`{raw}` must not contain `..`"), Component::RootDir | Component::Prefix(_) => { bail!("`{raw}` must be relative to the workspace root") } } } Ok(Self(path.to_path_buf())) } } /// Counts the lines of a file in the workspace. /// /// # Arguments /// - `path`: File path relative to the workspace root. #[swiftide::tool] pub async fn count_lines(context: &dyn AgentContext, path: &str) -> Result { let path: WorkspacePath = path.parse()?; let content = context .executor() .exec_cmd(&Command::read_file(&path.0)) .await?; Ok(ToolOutput::text( content.to_string_lossy().lines().count().to_string(), )) } ``` Prefer `Command::read_file` and `Command::write_file` over `cat` or `>` in a shell string: no shell ever sees the path. ## Docker executor For isolation, the [Docker executor](https://github.com/bosun-ai/swiftide-docker-executor) runs the commands in a container. It is a separate crate; pass it to `DefaultContext::from_executor`, and check that its swiftide version matches yours. ## Gotchas * `DefaultContext::from_executor` takes `Into>`. A bare `LocalExecutor` does not convert, so wrap it in an `Arc` annotated as `Arc`, as the scoped example does. * `LocalExecutor::stream_files` uses the path as given, not relative to `workdir`. A relative path resolves against your process’s current directory, also through a scope. Scope it with an absolute path or pass an absolute path. * Symlinks inside the working directory can still point elsewhere. Validated paths do not change that; a container does. ## Reference * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [`IndexingStream`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.IndexingStream.html) * [`Command`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.Command.html) * [`CommandOutput`](https://docs.rs/swiftide/0.34/swiftide/traits/struct.CommandOutput.html) * [`CommandOutputChunk`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.CommandOutputChunk.html) * [`CommandOutputSink`](https://docs.rs/swiftide/0.34/swiftide/traits/type.CommandOutputSink.html) * [`CommandError`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.CommandError.html) * [`ExecutorExt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ExecutorExt.html) * [`ScopedExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/struct.ScopedExecutor.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) * [`DefaultContext::from_executor`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html#method.from_executor) * [`AgentContext::executor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html#tymethod.executor) ## Next [Lifecycle hooks](/agents/hooks/) # Lifecycle hooks > Register agent hooks that log tool calls, change completion requests, rewrite tool output and report why an agent stopped. You end up with hooks that print each tool call, change a completion request before it goes out, trim tool output before the model reads it, and report why an agent stopped. A hook is a closure that the agent calls at a fixed point in its loop. The closure returns a boxed future that the agent awaits. Hooks need the `swiftide-agents` feature flag. The release-notes showcase prints one line per tool call and one per result: examples/release-notes-agent/src/progress.rs ```rust /// Prints each tool call and its result on one line. pub fn report_tool_calls(builder: &mut AgentBuilder) { builder .before_tool(|_agent, tool_call| { println!(" → {}", describe_call(tool_call)); Box::pin(async { Ok(()) }) }) .after_tool(|_agent, tool_call, result| { println!(" {} {}", tool_call.name(), describe_result(result)); Box::pin(async { Ok(()) }) }); } ``` ## Hook registration You pass a closure to an `AgentBuilder` method such as `before_tool`. The closure returns `Box::pin(async { Ok(()) })`, which resolves to `anyhow::Result<()>`. An `async` closure does not satisfy the hook traits. The closure must be `Clone`, `Send` and `Sync`. Share state through an `Arc`. Capturing a value that is not `Clone`, such as an `mpsc::Receiver`, fails to compile. Hooks of one kind run in the order you register them. `add_hook` appends a `Hook` value, and `hooks` replaces all earlier hooks. This is the start of a real run on the Swiftide repository, with `claude-sonnet-5-5` behind an OpenAI-compatible gateway: ```text Drafting release notes for the 7 commits since v0.33.0 with claude-sonnet-5-5 at http://127.0.0.1:8080/v1 (chat completions) → git_log_since v0.33.0 git_log_since ✓ 346 lines → git_diff swiftide-integrations/Cargo.toml 3b1ad22e^..3b1ad22e → git_diff …/src/chat_completion/chat_completion_response.rs 90bcadc7^..90bcadc7 → git_diff swiftide-agents-contrib/Cargo.toml 71fa4587^..71fa4587 → git_diff Cargo.toml 71fa4587^..71fa4587 → git_diff swiftide-integrations/src/aws_bedrock_v2/mod.rs 3b1ad22e^..3b1ad22e → git_diff swiftide/Cargo.toml 71fa4587^..71fa4587 git_diff ✓ 15 lines git_diff ✓ 72 lines git_diff ✓ 75 lines git_diff ✓ No changes to Cargo.toml in 71fa4587^..71fa4587. git_diff ✓ 18 lines git_diff ✓ 32 lines ``` The `→` lines come from `before_tool` and the `✓` lines from `after_tool`. The agent runs `before_tool` for a call and then starts that call, so the tool calls of one completion run concurrently. An earlier tool can already be running while `before_tool` runs for a later one. It then waits for the results and runs `after_tool` in call order. ## Hook points | Builder method | When it runs | What it receives | | ------------------- | --------------------------------------------------------------------------- | --------------------------------------------------------------- | | `before_all` | Once, on the first run, before the system prompt renders and toolboxes load | `&mut Agent` | | `on_start` | On every `query` or `run`, before its first completion | `&mut Agent` | | `before_completion` | Before each completion request | `&mut Agent`, `&mut ChatCompletionRequest` | | `on_stream` | For each stream chunk, when the agent streams | `&Agent`, `&ChatCompletionResponse` | | `after_completion` | After each completion, before the agent acts on it | `&mut Agent`, `&mut ChatCompletionResponse` | | `on_new_message` | When `Agent::add_message` adds a message | `&Agent`, `&mut ChatMessage` | | `before_tool` | Before each tool call starts | `&mut Agent`, `&ToolCall` | | `after_tool` | After each tool call, before its output is recorded | `&mut Agent`, `&ToolCall`, `&mut Result` | | `after_each` | After a completion and its tool calls | `&mut Agent` | | `on_stop` | When the agent records a stop reason | `&mut Agent`, `StopReason`, `Option<&AgentError>` | Since 0.33, most hooks receive `&mut Agent`, so they can stop the agent, read its history or change its tools: ```rust let mut agent = Agent::builder() .llm(&openai) .after_each(|agent: &mut Agent| { Box::pin(async move { if agent.history().await?.len() > 40 { agent.stop("history too long").await; } Ok(()) }) }) .build()?; ``` Only the `on_stream` builder method turns streaming on. A `Hook::OnStream` passed to `add_hook` or `hooks` never fires unless streaming is on. [Streaming](/agents/streaming/) covers stream chunks. ## Completion request and response edits ```rust let agent = Agent::builder() .llm(openai) .before_completion(|_agent, request| { println!( "sending {} messages and {} tools", request.messages().len(), request.tools_spec().len() ); // Only this request gets the reminder; the message history stays as it is. request .messages .to_mut() .push(ChatMessage::new_user("Reply in at most three sentences.")); Box::pin(async { Ok(()) }) }) .after_completion(|_agent, response| { if let Some(usage) = &response.usage { println!("used {} tokens", usage.total_tokens); } Box::pin(async { Ok(()) }) }) .build()?; ``` A change to `messages` or `tools_spec` applies to this one request. The reminder reaches the model but never enters the message history. Removing an entry from `tools_spec` hides a tool for one turn. In `after_completion`, changing `response.message` or `response.tool_calls` changes what the agent records and which tools it runs. `response.usage` is `None` when the provider reports no usage. ## Tool output rewrites ```rust const KEEP_LINES: usize = 200; let agent = Agent::builder() .llm(openai) .after_tool(|_agent, tool_call, result| { if let Ok(ToolOutput::Text(text)) = result { let lines: Vec<&str> = text.lines().collect(); if lines.len() > KEEP_LINES { let dropped = lines.len() - KEEP_LINES; let tail = lines[dropped..].join("\n"); *text = format!( "[{dropped} lines of {} output cut]\n{tail}", tool_call.name() ); } } Box::pin(async { Ok(()) }) }) .build()?; ``` Whatever you leave in the result is what the agent records in the message history and its tracing spans. The hook runs before the agent looks at errors. Replacing an `Err` with `Ok(ToolOutput::fail(..))` still tells the model what went wrong, but no longer counts toward `tool_retry_limit`. [Tool errors and retries](/agents/tools/#tool-errors-and-retries) explains the retry limit. ## Stop notifications ```rust let agent = Agent::builder() .llm(openai) .on_stop(|_agent, reason, error| { match error { Some(error) => eprintln!("agent failed: {error}"), None => println!("agent stopped: {reason:?}"), } Box::pin(async { Ok(()) }) }) .build()?; ``` `on_stop` runs once per stop and before the agent records its new state, so read the reason from the argument. [Stopping and structured results](/agents/stopping-and-structured-results/) lists every stop reason. `on_stop` is not a failure signal. The error is `Some` only when a completion step fails. Errors before the loop, such as a failed toolbox load, skip `on_stop`. A tool over `tool_retry_limit` fires `on_stop` with `StopReason::ToolCallsOverLimit` and no error, yet `query` returns `Err`. Check the result of `query` for failures. ## Hooks from methods ```rust /// Counts tool calls per tool, shared between the hook and the code that reads it. #[derive(Clone, Default)] pub struct ToolCallCounter { counts: Arc>>, } impl ToolCallCounter { /// `use<>` keeps `&self` out of the returned type, so the hook is `'static`. pub fn hook(&self) -> impl AfterToolFn + use<> { let counts = Arc::clone(&self.counts); move |_agent, tool_call, _result| { let mut counts = counts.lock().unwrap_or_else(PoisonError::into_inner); *counts.entry(tool_call.name().to_owned()).or_default() += 1; Box::pin(async { Ok(()) }) } } pub fn count(&self, tool_name: &str) -> usize { let counts = self.counts.lock().unwrap_or_else(PoisonError::into_inner); counts.get(tool_name).copied().unwrap_or_default() } } pub async fn count_tool_calls(openai: &OpenAI) -> anyhow::Result<()> { let counter = ToolCallCounter::default(); let mut agent = Agent::builder() .llm(openai) .after_tool(counter.hook()) .build()?; agent.query("List the files in this directory").await?; println!("stop was called {} times", counter.count("stop")); Ok(()) } ``` Builder methods only accept `'static` hooks. In Rust 2024, `fn hook(&self) -> impl AfterToolFn` captures the lifetime of `&self`. Clone the `Arc` into the closure and add `use<>` to the return type. ## Gotchas * The future may only borrow the agent. Touch the request, response, tool call or message before `Box::pin`. The `after_tool` result is the exception. * A hook that returns `Err` is logged with `tracing::error!` and the agent carries on. * `agent.stop(..)` lets the current step finish. A stop in `before_completion` still sends that request, and one in `before_tool` still runs the tool. In `before_all` and `on_start`, the agent fires `on_stop` but runs the loop anyway. * In `before_all`, `agent.tools()` does not yet hold Model Context Protocol (MCP) toolbox tools. It can fill template values through `system_prompt_mut`, as [System prompts](/agents/system-prompts/) shows. * `on_new_message` skips the system prompt and user queries, because the agent writes those to the context directly. * A resumed tool call that waited for approval runs again, so a counter like the one above counts it twice. Its tool hooks run before `on_start` while the agent is still stopped, so a stop from them is ignored. [Human in the loop](/agents/human-in-the-loop/) covers approvals. * A call to a tool the agent does not have is skipped, and no tool hook runs for it. ## Reference * [`hooks` module](https://docs.rs/swiftide/0.34/swiftide/agents/hooks/index.html) * [`Hook`](https://docs.rs/swiftide/0.34/swiftide/agents/hooks/enum.Hook.html) * [`AfterToolFn`](https://docs.rs/swiftide/0.34/swiftide/agents/hooks/trait.AfterToolFn.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`ChatCompletionRequest`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionRequest.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`ToolError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.ToolError.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) * [Feature flags](/reference/feature-flags/) ## Next [System prompts](/agents/system-prompts/) # Human in the loop > Wrap a tool in ApprovalRequired so the agent stops with StopReason::FeedbackRequired, then approve or refuse the call and resume the agent with run() or query(). An agent can stop before a tool runs, wait for a person to approve or refuse the call, and then continue where it stopped. The agent types on this page live in `swiftide::agents`, behind the `swiftide-agents` feature flag. The example also uses the `openai` integration and the default `macros` feature flag: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents ``` ## Approval wrapper ```rust // ApprovalRequired is a simple wrapper. You can also implement your own approval // flows by returning a `ToolOutput::FeedbackRequired` in a tool, // you can then use `has_received_feedback` and `received_feedback` on the context // to build your custom workflow. let guess_with_approval = ApprovalRequired::new(guess_a_number()); ``` The comment in the example gets one method name wrong: the method on the context that stores feedback is `feedback_received`. ```rust let mut agent = agents::Agent::builder() .llm(&openai) .tools(vec![guess_with_approval]) // Every message added by the agent will be printed to stdout .on_new_message(move |_, msg| { println!("{msg}"); Box::pin(async move { Ok(()) }) }) .limit(5) .build()?; ``` Register the wrapper in place of the bare tool. [`ApprovalRequired`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.ApprovalRequired.html) wraps any tool. It keeps the inner tool’s name and spec, so the model sees the same tool. When the model calls it, the wrapper asks the context for feedback on that tool call: * No feedback yet: it returns `ToolOutput::FeedbackRequired(None)`. The agent stops with `StopReason::FeedbackRequired { tool_call, payload }` and records no tool output, so the tool call stays open. * Approved: it runs the inner tool and returns its output. * Refused: it returns the text `This tool call was refused`, which the model reads as the tool output. `query` returns `Ok(())` when the agent stops this way. Read the stop reason to find the call that waits: ```rust // First query the agent, the agent will stop with a reason that feedback is required agent .query("Guess a number between 0 and 100 using the `guess_a_number` tool") .await?; // The agent stopped, lets get the tool call let Some(StopReason::FeedbackRequired { tool_call, .. }) = agent.stop_reason() else { panic!("expected a tool call to approve") }; // Alternatively, you can also get the stop reason from the agent state // agent.state().stop_reason().unwrap().feedback_required().unwrap() // Register that this tool call is ok. println!("Approving number guessing"); agent .context() .feedback_received(tool_call, &ToolFeedback::approved()) .await .unwrap(); // Run the agent again and it will pick up where it stopped. agent.run().await.unwrap(); ``` `feedback_received` on the context stores a [`ToolFeedback`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.ToolFeedback.html) for that tool call. `run` adds no user message. Before it asks the model for anything, it invokes the tool calls of the last assistant message again, provided no tool output follows that message. This time the wrapper finds the approval and runs the inner tool, and the loop carries on as usual. ## Approval loop The release notes showcase wraps its `write_changelog` tool as `OneEntryPerChange::new(ApprovalRequired::new(WriteChangelog::new(..)))` and loops until a maintainer approves a draft or gives up: examples/release-notes-agent/src/session.rs ```rust /// Asks the agent for release notes and loops through review rounds until they are written or /// rejected. pub async fn draft_release_notes( agent: &mut Agent, task: Prompt, reviewer: &mut impl Reviewer, ) -> Result { agent.query(task).await?; let mut written = false; let mut nudged = false; loop { // `write_changelog` is wrapped in `ApprovalRequired`: the agent stops and waits. let tool_call = match agent.stop_reason() { Some(StopReason::FeedbackRequired { tool_call, .. }) => tool_call.clone(), // The custom `stop` tool carries the structured release notes. Some(StopReason::RequestedByTool(_, Some(notes))) if written => { let notes = ReleaseNotes::deserialize(notes) .context("the agent stopped with malformed release notes")?; return Ok(Outcome::Written { notes: Some(notes) }); } _ if written => return Ok(Outcome::Written { notes: None }), // A turn without a tool call, or the step limit. Some(StopReason::NoNewMessages) => { nudge_once(agent, &mut nudged).await?; continue; } other => bail!("the agent stopped without writing the notes: {other:?}"), }; let proposal: WriteChangelogArgs = serde_json::from_str(tool_call.args().unwrap_or_default())?; match reviewer.review(&proposal)? { Verdict::Approve => { agent .context() .feedback_received(&tool_call, &ToolFeedback::approved()) .await?; // Resumes the pending tool call, which now runs, and continues the loop. agent.run().await?; // The write itself can still fail; the agent then hears why and carries on. written |= succeeded(agent, &tool_call).await?; } Verdict::Revise(feedback) => { agent .context() .feedback_received(&tool_call, &ToolFeedback::refused()) .await?; // The refusal is recorded as the tool's output, followed by the feedback. agent.query(feedback).await?; } Verdict::Reject => return Ok(Outcome::Rejected), } } } ``` The arguments of the waiting call are the proposal. The session parses them into the arguments struct that `#[derive(Tool)]` generates and hands them to a `Reviewer`, a small trait with a terminal implementation in `terminal.rs`. After an approval the write can still fail, so the session checks the history for a `ToolOutput::Text` answering that call before it counts the notes as written. ## Captured approval This is the approval excerpt of a real run on the Swiftide repository. The model was `claude-sonnet-5-5` behind an OpenAI-compatible gateway, and `y` was piped into stdin, which the reviewer echoes: ```text → write_changelog <32 lines> write_changelog ⏸ waiting for approval ── proposed: 32 lines into a new file, RELEASE_NOTES.md Write these release notes? [y/N] y → write_changelog <32 lines> write_changelog ✓ wrote RELEASE_NOTES.md → stop (version 0.34.0: highlights 2, breaking 1, fixes 1) stop ■ done ``` The `→` lines come from a `before_tool` hook and the result lines from an `after_tool` hook. `⏸ waiting for approval` is the hook seeing `ToolOutput::FeedbackRequired`. The second `→ write_changelog` is the same tool call, invoked again by `run()` after the approval. ## Refusals and revisions A refusal is an answer, not an error. `ToolFeedback::refused()` makes the wrapper return its refusal text, and the agent records it as the tool output. `query(feedback)` queues the person’s message until that output is recorded, so the history reads: tool call, refusal, feedback. The model then drafts again and calls `write_changelog` with a new tool call, which starts a new approval round. Neither `approved()` nor `refused()` takes an argument. To attach a payload, chain `.with_payload(serde_json::Value)` after the constructor. `ApprovalRequired` ignores it; your own tools can read it. ## Draft checks `OneEntryPerChange` in the showcase’s `tools.rs` wraps the approval tool. It parses the draft and returns `ToolOutput::fail` when one pull request or commit appears in more than one entry; otherwise it calls the wrapped tool. The model reads the failure and fixes the draft, and the maintainer only sees drafts that pass. A `ToolOutput::fail` does not count toward the tool retry limit ([Tool errors and retries](/agents/tools/#tool-errors-and-retries)). Put checks outside the approval wrapper, so they run before anyone is asked. ## Custom feedback tools A tool can ask for feedback itself by returning `ToolOutput::feedback_required(Some(payload))`. The payload reaches the caller as the `payload` of `StopReason::FeedbackRequired`. The tool needs the `ToolCall` to look up feedback, so it implements [`Tool`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/trait.Tool.html) by hand; `#[swiftide::tool]` functions and `#[derive(Tool)]` structs only receive the context and their arguments. ```rust #[derive(Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] struct RefundArgs { /// The order to refund. order_id: String, /// The amount to refund, in cents. amount_cents: u64, } #[derive(Clone)] struct IssueRefund; #[swiftide::reexports::async_trait::async_trait] impl Tool for IssueRefund { async fn invoke( &self, context: &dyn AgentContext, tool_call: &ToolCall, ) -> Result { let args = tool_call .args() .ok_or_else(|| ToolError::missing_arguments("order_id, amount_cents"))?; let refund: RefundArgs = serde_json::from_str(args)?; match context.has_received_feedback(tool_call).await { // No decision yet: stop the agent and show the reviewer what it asks for. None => Ok(ToolOutput::feedback_required(Some(json!({ "order_id": refund.order_id, "amount_cents": refund.amount_cents, })))), // The reviewer can lower the amount in the approval payload. Some(ToolFeedback::Approved { payload }) => { let amount_cents = payload .as_ref() .and_then(|payload| payload["amount_cents"].as_u64()) .unwrap_or(refund.amount_cents); send_refund(&refund.order_id, amount_cents).await?; Ok(ToolOutput::text(format!( "Refunded {amount_cents} cents for order {}", refund.order_id ))) } Some(ToolFeedback::Refused { payload }) => { let reason = payload .as_ref() .and_then(|payload| payload["reason"].as_str()) .unwrap_or("no reason given"); Ok(ToolOutput::text(format!( "A person refused the refund: {reason}" ))) } } } fn name(&self) -> Cow<'_, str> { Cow::Borrowed("issue_refund") } fn tool_spec(&self) -> ToolSpec { ToolSpec::builder() .name("issue_refund") .description("Refund (part of) an order. A person approves every refund.") .parameters_schema(schema_for!(RefundArgs)) .build() .expect("the refund arguments form a strict schema") } } ``` `DefaultContext` removes the feedback when `has_received_feedback` returns it, so each decision applies to one invocation. The caller can change what the tool does through the payload of the approval: ```rust if let Some(StopReason::FeedbackRequired { tool_call, payload }) = agent.stop_reason() { println!("approve {payload:?}?"); let tool_call = tool_call.clone(); let feedback = ToolFeedback::approved().with_payload(json!({ "amount_cents": 2_500 })); agent .context() .feedback_received(&tool_call, &feedback) .await?; agent.run().await?; } ``` ## Gotchas * Have the model call an approval tool on its own. `run` and `query` only resume open tool calls when no tool output follows the assistant message. If the same completion also called another tool, that tool’s output is recorded and the approved call never runs. The showcase prompt says “Call `write_changelog` with the Markdown, on its own.” * The stop reason names only the first tool call that asked for feedback in a completion. * Calling `run()` or `query()` before giving feedback invokes the wrapper again, which asks again, and the agent returns `AgentError::UnfulfilledToolCalls`. A message passed to that `query` stays queued, and the next successful `run()` or `query()` adds it to the history after the tool output. Retry with `run()` so the message does not appear twice. * Hooks see the resumed call twice ([Lifecycle hooks](/agents/hooks/#gotchas)). * Feedback lives in the context, not the message history. To approve in another process, see [Context and history](/agents/context-and-history/). ## Reference * [`ApprovalRequired`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.ApprovalRequired.html) * [`ToolFeedback`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.ToolFeedback.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) (`query`, `run`, `stop_reason`, `context`) * [`AgentContext`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html) (`feedback_received`, `has_received_feedback`) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) (`with_tool_feedback`) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`Tool`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/trait.Tool.html), [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html), [`ToolSpec`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolSpec.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) ## Next [Streaming](/agents/streaming/) # MCP tools > Start a Model Context Protocol (MCP) server as a child process, wrap it in an McpToolbox with a whitelist, and add its tools to a Swiftide agent. Here you give an agent a few tools from a Model Context Protocol (MCP) server that runs as a child process and hide the rest from the model. Swiftide speaks MCP through the [`rmcp`](https://docs.rs/rmcp/2.1) crate. Turn on the `mcp` feature flag, which also enables `swiftide-agents`, and add `rmcp` with a client and the transport you use: ```sh cargo add swiftide@0.34 --features openai,mcp cargo add rmcp@2.1 --features client,transport-child-process cargo add tokio --features macros,rt-multi-thread,process,time ``` ## Child process toolbox The release-notes-agent showcase reads pull requests through the GitHub MCP server when you pass `--mcp`: examples/release-notes-agent/src/mcp.rs ```rust /// Starts `@modelcontextprotocol/server-github` with `npx` and exposes a few of its tools. /// /// Errors explain what is missing, so the caller can carry on without GitHub. pub async fn github_toolbox() -> Result { let token = env::var("GITHUB_PERSONAL_ACCESS_TOKEN") .or_else(|_| env::var("GITHUB_TOKEN")) .context("set GITHUB_PERSONAL_ACCESS_TOKEN (or GITHUB_TOKEN)")?; let server = Command::new("npx").configure(|cmd| { cmd.args(["-y", "@modelcontextprotocol/server-github"]) .env("GITHUB_PERSONAL_ACCESS_TOKEN", token); }); let (transport, _stderr) = TokioChildProcess::builder(server) .stderr(Stdio::null()) .spawn() .context("could not run `npx`; is Node.js installed?")?; let mut toolbox = timeout( Duration::from_mins(1), McpToolbox::try_from_transport(transport), ) .await .context("the GitHub MCP server did not start within a minute")? .context("the GitHub MCP server failed to start")?; toolbox.with_whitelist(GITHUB_TOOLS); // Swiftide names MCP tools `:`; OpenAI rejects ':', see `OpenAiToolNames` below. let toolbox = OpenAiToolNames(toolbox); // Fail early, rather than mid-run, if the server does not offer the tools we expect. let available = toolbox.available_tools().await?; if available.len() != GITHUB_TOOLS.len() { bail!("the GitHub MCP server does not offer {GITHUB_TOOLS:?}"); } Ok(toolbox) } ``` `McpToolbox::try_from_transport` takes any `rmcp` client transport, such as a child process or a Transmission Control Protocol (TCP) stream. It performs the MCP handshake with a default client info named `swiftide` and returns the toolbox. The handshake waits for `npx` to download the package on a first run, so the showcase puts a one-minute timeout around it. Calling `available_tools` up front turns a missing tool into a startup error instead of a failure in the middle of a run. `src/agent.rs` adds the toolbox with `builder.add_toolbox(github_tools)`. Run it from the swiftide repository. You need `OPENAI_API_KEY`, Node.js, a token in `GITHUB_PERSONAL_ACCESS_TOKEN` or `GITHUB_TOKEN`, and an `origin` remote on GitHub. Without the remote, the showcase skips the GitHub tools: ```sh cargo run -p release-notes-agent -- --mcp ``` ## Running service toolbox When you need your own client info or capabilities, start the `rmcp` service yourself and hand it over: ```rust // First set up our client info to identify ourselves to the server let client_info = ClientInfo::new( ClientCapabilities::default(), Implementation::new("swiftide-example", env!("CARGO_PKG_VERSION")), ); // Use `rmcp` to start the server let running_service = client_info .serve(TokioChildProcess::new( tokio::process::Command::new("npx").configure(|cmd| { cmd.args(["-y", "@modelcontextprotocol/server-everything"]); }), )?) .await?; // Create a toolbox from the running server, and only use the `add` tool // // A toolbox reveals it's tools to the swiftide agent the first time it starts (if the state of // the agent was pending). You can add as many toolboxes as you want. MCP services are an // implmenentation of a toolbox. A list of tools is another. let everything_toolbox = McpToolbox::from_running_service(running_service) .with_whitelist(["add"]) .to_owned(); ``` `ClientInfo::serve` comes from `rmcp`’s `ServiceExt` and returns a `RunningService`. `McpToolbox::from_running_service` wraps it without contacting the server. Serve from a `ClientInfo`, not from `()`. The filter methods return `&mut Self`, so finish a chain on a temporary with `.to_owned()`. ```rust agents::Agent::builder() .llm(&openai) // Add the toolbox to the agent .add_toolbox(everything_toolbox) // Every message added by the agent will be printed to stdout .on_new_message(move |_, msg| { let msg = msg.to_string(); let tx = tx.clone(); Box::pin(async move { tx.send(msg).unwrap(); Ok(()) }) }) .build()? .query("Use the add tool to add 1 and 2") .await?; ``` `add_toolbox` accepts any owned `ToolBox + 'static`. You can add as many toolboxes as you like. A `Vec>` is a toolbox too. The agent lists a toolbox’s tools when it first starts, after the `before_all` hooks and the system prompt, not when you call `build`. If listing fails, `query` returns `AgentError::ToolBoxFailedToLoad`. After a successful listing the agent keeps the list, so tools the server adds later do not reach it. A failed listing is retried on the next `query`, and the `before_all` hooks run again. The example needs `OPENAI_API_KEY` and Node.js: ```sh cargo run -p swiftide-examples --example agents-mcp ``` It sends the prefixed name `:add` to OpenAI unchanged. If OpenAI rejects it, wrap the toolbox as described under [Tool names](/agents/mcp/#tool-names). ## Filters A toolbox holds one filter, and each call replaces it: * `with_whitelist(names)` keeps only the named tools. * `with_blacklist(names)` keeps every tool except the named ones. * `with_filter(ToolFilter::Whitelist(..))` or `with_filter(ToolFilter::Blacklist(..))` sets a filter built elsewhere, such as from configuration. `ToolFilter` implements `Serialize` and `Deserialize`. Names match the tool names the server reports, exactly and without the server prefix. A whitelisted name that the server does not offer is skipped without an error. A blacklist suits a server where most tools are safe to expose. This toolbox keeps the GitHub server’s read tools and hides three that write: ```rust // `github` is an `McpToolbox` from `try_from_transport` or `from_running_service`. github .with_name("github") .with_blacklist(["create_or_update_file", "push_files", "merge_pull_request"]); let agent = Agent::builder() .llm(openai) .add_toolbox(github) .build()?; ``` `with_name` and `with_blacklist` both return `&mut Self`, so they chain on a `let mut` binding, and the toolbox itself then moves into `add_toolbox`. A tool that a new version of the server adds passes a blacklist, but not a whitelist. ## Tool names Since 0.32, Swiftide registers each MCP tool as `:`. The server name comes from the handshake, and falls back to `mcp` when it is missing or empty. The model sees the prefixed name, and Swiftide calls the server with the original one. `with_name` only names the toolbox in logs and does not change the prefix. OpenAI accepts tool names made of letters, digits, `_` and `-`, so it rejects the `:`. Swiftide does not rewrite the names. The showcase wraps its toolbox in `OpenAiToolNames`, a `ToolBox` in `src/mcp.rs` that replaces every other character with `_`. The model then sees `github-mcp-server_get_pull_request`. Tool names are unique per agent. When a toolbox tool has the same name as a tool the agent already has, the agent keeps the existing one. ## Results Since 0.33, Swiftide reads the content blocks of `rmcp` 2 tool results: * Text blocks are joined with newlines into a text tool output. Other block kinds, such as images, are dropped. * Only a result with no content blocks at all uses its `structured_content`, serialized as JSON (JavaScript Object Notation). * A successful result with no text left becomes `Tool executed successfully`. That includes a result with only image blocks, even when it carries `structured_content`. * A result with `is_error` set becomes `ToolError::Unknown` with the server’s text, or `Unknown error` when there is none. The model reads the error as the tool output, and the failure counts toward the tool retry limit (`tool_retry_limit`). See [Tool errors and retries](/agents/tools/#tool-errors-and-retries). * Arguments that are not valid JSON return `ToolError::WrongArguments` before the server is called. ## Shutdown `McpToolbox::cancel` disconnects from the server and stops the `rmcp` service. A toolbox is `Clone`, and clones share one connection with the tools they listed. Bind a clone with `let mut` before you pass the toolbox to `add_toolbox`, and call `cancel().await` on it once the agent is done. After `cancel`, calls to the server’s tools return a tool error that says the server is not running, and listing the tools fails. A second `cancel` logs a warning and returns `Ok`. ## Gotchas * Swiftide supports MCP tools only. Prompts, resources and sampling from the server are not used. * Use `rmcp` 2. Another major version brings its own `RunningService` type, which `from_running_service` does not accept. * Every tool that passes the filter needs a description, and its input schema must parse as a JSON Schema that Swiftide accepts. Otherwise `query` returns `AgentError::ToolBoxFailedToLoad` for the whole toolbox, so filter out the tools you do not need. * A whitelist limits what the model can call, not what the server can do. Pick read-only tools when the model’s arguments are untrusted, as the showcase does. ## Reference * [`McpToolbox`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/mcp/struct.McpToolbox.html) * [`ToolFilter`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/mcp/enum.ToolFilter.html) * [`AgentBuilder::add_toolbox`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.add_toolbox) * [`ToolBox`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolBox.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) * [`ToolError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.ToolError.html) * [`rmcp` transports](https://docs.rs/rmcp/2.1/rmcp/transport/index.html) ## Next [Providers and reasoning](/agents/providers-and-reasoning/) # How agents work > Build a Swiftide agent, run its completion and tool-call loop, and read why it stopped. After this page you can build an agent, run it, continue its conversation, and read why it stopped. The larger example is the [release-notes-agent showcase](/showcases/release-notes-agent/), which drafts release notes from git history. Agents live in `swiftide::agents`, behind the `swiftide-agents` feature flag: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents cargo add tokio --features full cargo add anyhow ``` ## A minimal agent ```rust // Reads OPENAI_API_KEY from the environment. let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .build()?; let mut agent = Agent::builder() .llm(&openai) .system_prompt("You answer questions about Rust crates in two sentences.") .limit(10) .build()?; // A failed run, including a tool that fails too often, returns `Err` here. agent.query("What does the tokio crate do?").await?; assert!(agent.is_stopped()); match agent.stop_reason() { Some(StopReason::RequestedByTool(..)) => println!("a tool stopped the agent"), Some(StopReason::NoNewMessages) => println!("no new messages, or the limit was reached"), other => println!("stopped: {other:?}"), } // A second query appends to the same message history and runs the loop again. agent.query("And how does it compare to async-std?").await?; println!("{} messages in the history", agent.history().await?.len()); ``` `OpenAI::builder()` reads `OPENAI_API_KEY` from the environment. `llm` takes any `ChatCompletion` that is `Clone` and clones it into the agent. `build` fails without it. `system_prompt` takes anything that converts into a `Prompt`. The agent has no system prompt unless you set one. `limit` caps the completions in one run. `query` adds a user message and loops until the agent stops. A stopped agent keeps its message history, so the second `query` continues the same conversation. `history` returns every message in the context. ## The agent loop ![Agent loop](/diagrams/diagram-agent-loop.svg) The agent sends the message history and the spec of every tool to the model. When the completion holds tool calls, the agent runs the tools and records their outputs in the history. Then it asks the model again. Hooks run around every step. The loop ends when a tool stops the agent, or when the step limit is reached. With `DefaultContext`, it also ends when the last message is an answer without a tool call. `DefaultContext::with_stop_on_assistant(false)` turns that off. A tool that returns `ToolOutput::FeedbackRequired` stops the agent and leaves its tool call open, without an output. A call to a tool the agent does not have is skipped with a warning and gets no output either. ## State and stop reasons `agent.state()` returns a `State`: `Pending` before the first run, `Running` during one, and `Stopped` with a `StopReason` after it. `agent.stop_reason()` tells you why it stopped: * a tool returned `ToolOutput::Stop` (`RequestedByTool`, which holds that tool call) * an answer without a tool call, the step limit, or the end of a `query_once` or `run_once` step (`NoNewMessages`) * a pending approval (`FeedbackRequired`) * a failure (`ToolCallsOverLimit`, `AgentFailed`, `Error`) For `ToolCallsOverLimit` and `Error`, `query` also returns the error, so the `?` in the snippet returns before the `match`. `StopReason` is marked non-exhaustive, so a `match` on it needs a wildcard arm. [Stopping and structured results](/agents/stopping-and-structured-results/) covers each reason. ## Runs `query_once` adds the message and runs one step. `run` and `run_once` do the same without a new message, which is how you resume after an approval. A run on an agent whose state is `Running` returns `AgentError::AlreadyRunning`. Cloning an agent shares its hooks and context, so both clones write to the same message history. Each clone gets its own copy of the tools, so `add_tool` on one does not change the other. The clone copies the state and starts with an empty tool retry counter. To run an agent as one step of a larger workflow, wrap it in a task node; see [Agents as task nodes](/tasks/agents-as-nodes/). ## A larger builder The release-notes-agent showcase builds its agent in one function: examples/release-notes-agent/src/agent.rs ```rust /// Builds the agent. All tools run through `executor`, whose working directory is the repo. pub fn build( llm: &(impl ChatCompletion + Clone + 'static), executor: Arc, task: &ReleaseTask, github_tools: Option, ) -> Result { let mut builder = Agent::builder(); builder .llm(llm) .context(DefaultContext::from_executor(executor)) .system_prompt(task.system_prompt(github_tools.is_some())) // Our own `stop` tool replaces the default one; call this before `tools`. .without_default_stop_tool() .tools([ git_log_since(), git_diff(), read_file(), // Drafts that describe a change twice go back to the model before anyone reviews them. OneEntryPerChange::new(ApprovalRequired::new(WriteChangelog::new(&task.notes_file))) .into(), ReleaseNotes::stop_tool().into(), ]) .limit(STEP_LIMIT); if let Some(github_tools) = github_tools { builder.add_toolbox(github_tools); } progress::report_tool_calls(&mut builder); progress::stream_draft(&mut builder); Ok(builder.build()?) } ``` The showcase passes an `OpenAI` client pointed at any OpenAI-compatible endpoint, but the function accepts any provider. `context` sets where the message history lives and which tool executor runs commands. `DefaultContext::from_executor` keeps the history in memory and uses the executor that `run` creates: examples/release-notes-agent/src/main.rs ```rust let root = Git::new(&LocalExecutor::new(&cli.repo)) .top_level() .await .with_context(|| format!("{} is not a git repository", cli.repo.display()))?; // Every tool the agent runs, runs with the repository as its working directory. let executor: Arc = Arc::new( LocalExecutor::builder() .workdir(&root) .default_timeout(Some(Duration::from_mins(1))) .build()?, ); ``` `Agent::builder()` starts with one tool, `stop`, which the model calls when it is done. The showcase replaces it with a `stop` tool whose arguments are the release notes, so it calls `without_default_stop_tool` before `tools`. The list mixes tools from `#[swiftide::tool]` functions, a tool wrapped in `ApprovalRequired`, and a `StopWithArgs` built from a JavaScript Object Notation (JSON) schema. `add_toolbox` adds the GitHub tools from a Model Context Protocol (MCP) server when you pass `--mcp` in a repository hosted on GitHub. The MCP toolbox needs the `mcp` feature flag, which the showcase enables. The two `progress` functions take `&mut AgentBuilder` and register hooks, which keeps the builder short. The [hooks page](/agents/hooks/) shows a captured run of this agent with `claude-sonnet-5-5`. ### Agent names `AgentBuilder::name` labels an agent, and `Agent::name()` reads the label back: ```rust let agent = Agent::builder() .llm(openai) .name("release-checker") .build()?; assert_eq!(agent.name(), "release-checker"); // Without `name`, every agent is called "unnamed_agent". let unnamed = Agent::builder().llm(openai).build()?; assert_eq!(unnamed.name(), "unnamed_agent"); ``` The name defaults to `unnamed_agent`. Swiftide only shows it in the agent’s `Debug` output, so use it in your own logs or to tell agents apart in a multi-agent setup. ## Agent and application responsibilities Besides running the loop, the agent: * renders the system prompt on the first run and appends it to the history, after any messages that `BeforeAll` hooks or an existing history put there * removes duplicate keys from the JSON arguments of a tool call * wraps queries, steps, tools and hooks in tracing spans ([Tracing](/observability/tracing/)) * lists toolbox tools when it first starts You provide the model, the tools, the system prompt, the tool executor, the hooks and, when you want a typed result, the schema of the stop tool. ## Gotchas * A hook that returns an error is logged with tracing. The agent carries on. * When a completion holds only calls to tools the agent does not have, no output is recorded. The next `query` returns `AgentError::UnfulfilledToolCalls`. * Some errors return from `query` without setting a stop reason, for example a message history error or a tool task that fails to join. The agent then stays `Running`, and the next run returns `AgentError::AlreadyRunning`. * A clone of a `Pending` agent renders the system prompt again on its first run and appends it to the shared history. ## Reference * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`State`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.State.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) * [`StopWithArgs`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.StopWithArgs.html) * [`ApprovalRequired`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.ApprovalRequired.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) ## Next [Tools](/agents/tools/) # Providers and reasoning > Run one agent definition on OpenAI, Azure, Anthropic, Ollama, AWS Bedrock or any OpenAI-compatible gateway, keep reasoning items across turns, and know how each provider treats tool schemas. You end up with one agent definition that runs on whichever provider you pick at startup, with the model’s reasoning carried from one turn to the next. ```rust /// Builds the same agent for any chat completion client. fn changelog_agent(llm: &(impl ChatCompletion + Clone + 'static)) -> anyhow::Result { let agent = Agent::builder() .llm(llm) .system_prompt("You summarize git history for release notes.") .limit(20) .build()?; Ok(agent) } ``` ```rust let mut agent = match std::env::var("LLM_PROVIDER").as_deref() { Ok("anthropic") => changelog_agent( &Anthropic::builder() .default_prompt_model("claude-sonnet-4-5") .build()?, )?, Ok("ollama") => changelog_agent(&Ollama::builder().default_prompt_model("qwen3").build()?)?, Ok("azure") => { let config = AzureConfig::new() .with_api_base("https://my-resource.openai.azure.com") .with_api_version("2024-10-21") .with_deployment_id("gpt-4o-mini") .with_api_key(std::env::var("AZURE_OPENAI_API_KEY")?); let azure: GenericOpenAI = GenericOpenAIBuilder::default() .client(async_openai::Client::with_config(config)) .default_prompt_model("gpt-4o-mini") .build()?; changelog_agent(&azure)? } _ => changelog_agent( &OpenAI::builder() .default_prompt_model("gpt-5-mini") .build()?, )?, }; agent .query("Summarize the commits since the last tag.") .await?; ``` `AgentBuilder::llm` takes a reference to any client that implements `ChatCompletion + Clone + 'static`. It stores a boxed clone, so `Agent` has no type parameter for the provider. The tools, hooks and system prompt stay the same whichever client you pass. The Azure client needs the `async-openai` crate as a direct dependency, because Swiftide does not re-export its client type. `api_base` and `api_key` on the builder only configure the `OpenAI` client, so Azure passes a full client to `client(..)`: ```sh cargo add async-openai@0.41 --no-default-features ``` Use the same version Swiftide depends on, or the client types do not match. ## Chat providers Each chat client sits behind its own feature flag. The snippet above needs `swiftide-agents` and three provider flags: ```sh cargo add swiftide@0.34 --features swiftide-agents,openai,anthropic,ollama ``` Every chat client implements `ChatCompletion`. [Providers](/models/overview/#providers) lists each client with its feature flag, and [OpenAI-compatible endpoints](/models/overview/#openai-compatible-endpoints) covers gateways and base URLs. The ask-your-codebase showcase builds its client from a base URL, a key and a flag for the Responses API, so the same binary runs against OpenAI or a gateway. It sets them with `api_base` and `api_key`: examples/ask-your-codebase/src/llm.rs ```rust impl LlmEndpoint { /// Builds a client for this endpoint. /// /// Rate limits and transient errors are retried with exponential backoff, which matters when /// indexing fires many prompts concurrently. After `retry_budget` the error is returned, so /// an unreachable or broken endpoint fails instead of hanging. pub fn client(&self, retry_budget: Duration) -> Result> { let openai = OpenAI::builder() .api_base(&self.base_url) .api_key(&self.api_key) .default_prompt_model(&self.model) .use_responses_api(matches!(self.api, OpenAiApi::Responses)) .build()?; let backoff = BackoffConfiguration { max_elapsed_time_sec: retry_budget.as_secs(), ..BackoffConfiguration::default() }; Ok(Arc::new(LanguageModelWithBackOff::new(openai, backoff))) } } ``` ## Bedrock agents `AwsBedrock` talks to the Bedrock Converse API. It reads AWS credentials and the region from your usual Amazon Web Services (AWS) configuration, such as a command-line interface (CLI) profile or environment variables: ```rust let bedrock = AwsBedrock::builder() .default_prompt_model("global.anthropic.claude-sonnet-4-6") .build()?; ``` The model ID here is a global inference profile. The agent itself looks like any other: ```rust let mut agent = agents::Agent::builder() .llm(&bedrock) .tools(vec![current_utc_time(), format_timestamp()]) .on_new_message(|_, msg| { let rendered = msg.to_string(); Box::pin(async move { println!("{rendered}"); Ok(()) }) }) .limit(6) .build()?; ``` Setup and credentials are on [AWS Bedrock](/integrations/aws-bedrock/). ## Reasoning items On `OpenAI`, reasoning models return their reasoning only through the Responses API. `AwsBedrock` returns signed Claude reasoning as `ReasoningItem` values without extra setup. Reasoning on `OpenAI` needs the Responses API and a reasoning effort; [Responses API and reasoning](/models/overview/#responses-api-and-reasoning) covers the client options. ```rust // Reasoning models require the Responses API. Enabling reasoning effort also asks for a // summary and encrypted reasoning content (enabled by default). If your OpenAI org is not // verified for reasoning access, summaries may be absent. Disable with // `reasoning_features(false)` if desired. let openai = OpenAI::builder() .default_prompt_model("o3-mini") .default_options(Options::builder().reasoning_effort(ReasoningEffort::Low)) .use_responses_api(true) .build()?; ``` Each `ReasoningItem` has a `summary` list and an optional `encrypted_content`. This helper joins the summaries: ```rust fn reasoning_summary(reasoning: Option<&[ReasoningItem]>) -> Option { let summary = reasoning .unwrap_or(&[]) .iter() .flat_map(|item| item.summary.iter()) .cloned() .collect::>() .join("\n"); if summary.is_empty() { None } else { Some(summary) } } ``` The agent reads them in an `after_completion` hook: ```rust let mut agent = Agent::builder() .llm(&openai) .on_new_message(|_, message| { if let ChatMessage::Assistant(content, _) = message && let Some(content) = content.as_deref() { println!("Assistant: {content}"); } Box::pin(async move { Ok(()) }) }) .after_completion(|_, response| { if let Some(summary) = reasoning_summary(response.reasoning.as_deref()) { println!("Reasoning summary:\n{summary}"); } let has_encrypted = response .reasoning .as_ref() .is_some_and(|items| items.iter().any(|item| item.encrypted_content.is_some())); println!("Encrypted reasoning content present: {has_encrypted}"); Box::pin(async move { Ok(()) }) }) .build()?; ``` After the `after_completion` hooks run, the agent adds every item in `response.reasoning` to the message history as a `ChatMessage::Reasoning`. It adds them before the assistant message, through `Agent::add_message`, so `on_new_message` hooks see them too. A hook that empties `response.reasoning` keeps the items out of the history. ## Reasoning replay What happens to those items on the next request depends on the client: * The Responses API client replays them only when `reasoning_effort` is set on that client. It then sends back every item with a non-empty `encrypted_content` and sets `store` to false, so the reasoning lives in your message history and survives a saved and restored history (see [Context and history](/agents/context-and-history/)). Without `reasoning_effort`, it drops every `ChatMessage::Reasoning`. * `AwsBedrock` replays an item only when it has non-empty `content` and a non-empty `encrypted_content`, which it sends as the signature. It drops anything else. * The Chat Completions client and `Anthropic` leave `ChatMessage::Reasoning` out of their requests. ## Bedrock reasoning effort `AwsBedrock` has its own `Options` and its own `ReasoningEffort` enum in `swiftide::integrations::aws_bedrock_v2`. Set `reasoning_effort` in those options and pass them to `default_options` on the builder. For the Opus 4.5 beta header and the other fields, see [AWS Bedrock settings](/models/overview/#aws-bedrock-settings). ## Tool schema policies `ToolSpec::builder().build()` checks every schema against Swiftide’s strict contract (see [Tools](/agents/tools/)) and rewrites what it can: a nullable type such as `["string", "null"]` becomes an `anyOf` with a null branch, a two-entry `oneOf` with a null branch becomes an `anyOf`, annotations next to a `$ref` are dropped, and every object gets `additionalProperties: false`. Each provider then applies its own policy when it builds a request. Only `AwsBedrock` lets you change it. * `OpenAI` and the other `GenericOpenAI` clients remove `$schema` and Rust integer formats such as `int32`, and list every property in `required`. They reject any remaining `oneOf` or array-valued `type`, and send the schema with `strict: true`. * `Anthropic` and `AwsBedrock` send the strict schema unchanged. `AwsBedrock` marks tools strict unless you set `tool_strict` to `false` in its `Options`. ## Gotchas * The OpenAI schema policy runs when the request is built, not when the tool is built. A schema it rejects fails the first completion with `AgentError::CompletionsFailed`, wrapping a permanent error that says `tool schema must be OpenAI compatible`. * Gateways do not always speak the Responses API. Against the gateway used for the showcase runs, the Responses API fails because the gateway streams events with `"output": null`, which the client cannot parse. Use it with OpenAI itself. * OpenAI rejects the `:` in Model Context Protocol (MCP) tool names; see [Tool names](/agents/mcp/#tool-names). * Reasoning items are tied to the provider that made them. Bedrock items carry a Bedrock signature in `encrypted_content`, and an OpenAI Responses client with `reasoning_effort` set sends that to OpenAI as encrypted reasoning. OpenAI items usually have no `content`, so `AwsBedrock` drops them silently. Remove `ChatMessage::Reasoning` entries from the message history before you switch providers. * `AwsBedrock::builder().build()` loads the AWS configuration with `block_in_place`, so it needs a multi-threaded Tokio runtime. A default `#[tokio::test]` or `flavor = "current_thread"` panics. Pass your own client with `.client(..)` otherwise. * A client’s `on_usage` callback counts every call on that client, including other agents that share it; see [Usage callbacks](/models/errors-backoff-and-usage/#usage-callbacks). ## Reference * [`AgentBuilder::llm`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.llm) * [`AgentBuilder::after_completion`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.after_completion) * [`AgentBuilder::on_new_message`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.on_new_message) * [`Agent::add_message`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html#method.add_message) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`GenericOpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html) * [`GenericOpenAIBuilder::use_responses_api`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.use_responses_api) * [`Options` (OpenAI)](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.Options.html) * [`AzureConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.AzureConfig.html) * [`Anthropic`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.Anthropic.html) * [`Ollama`](https://docs.rs/swiftide/0.34/swiftide/integrations/ollama/struct.Ollama.html) * [`AwsBedrock`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrock.html) * [`Options` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.Options.html) * [`ReasoningEffort` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/enum.ReasoningEffort.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`ReasoningItem`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ReasoningItem.html) * [`ToolSpec`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolSpec.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) ## Next [Contrib tools and hooks](/agents/contrib/) # Stopping and structured results > Replace the stop tool with StopWithArgs and a JSON schema to get a typed result, report failures with AgentCanFail, cap runs with limit and tool_retry_limit, and handle every StopReason. Your agent ends its run by handing your code a typed value, such as `ReleaseNotes`, or a typed report of why it failed. Your code can also tell why any other run ended. `Agent::builder()` starts with one tool, named `stop`, which takes no arguments and ends the run with `StopReason::RequestedByTool(tool_call, None)`. With the default context, an answer without a tool call also ends the run, with `StopReason::NoNewMessages`. ## Structured results The examples need the `swiftide-agents` feature flag and your own `schemars` 1.x, `serde` and `serde_json`: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents cargo add schemars@1 serde serde_json --features serde/derive ``` The [release-notes-agent showcase](/showcases/release-notes-agent/) returns its notes as data. It describes them as a type and turns the type’s JSON (JavaScript Object Notation) Schema into a stop tool: examples/release-notes-agent/src/notes.rs ```rust /// Structured release notes. The agent hands these over when it calls `stop`. #[derive(Debug, Serialize, Deserialize, JsonSchema)] #[schemars(deny_unknown_fields)] pub struct ReleaseNotes { /// The version being released, without a leading `v`. pub version: String, /// One or two sentences on what this release is about, written for users. pub summary: String, /// New features and notable improvements. pub highlights: Vec, /// Changes that require users to update their code. Empty if there are none. pub breaking_changes: Vec, /// Bug fixes users may notice. pub fixes: Vec, } #[derive(Debug, Serialize, Deserialize, JsonSchema)] #[schemars(deny_unknown_fields)] pub struct Item { /// Short, user-facing title. pub title: String, /// What changed and why it matters, in one or two sentences. pub details: String, /// Pull requests or commits as plain text, e.g. `#1207` or `90bcadc7`, not Markdown links. pub references: Vec, } impl ReleaseNotes { /// A `stop` tool whose arguments must match [`ReleaseNotes`]. /// /// With a custom schema the arguments are forwarded as-is, and end up in /// `StopReason::RequestedByTool`. pub fn stop_tool() -> StopWithArgs { StopWithArgs::with_parameters_schema(schema_for!(ReleaseNotes)) } } ``` `StopWithArgs::with_parameters_schema` builds a tool that is also named `stop`. The model calls it with arguments that follow the schema. The doc comments on the fields end up in the schema, so the model reads them as instructions. `deny_unknown_fields` gives both objects `additionalProperties: false`. With a custom schema, the tool passes the whole argument object on as the payload. The agent stops with `StopReason::RequestedByTool(tool_call, Some(payload))`, where `payload` is a `serde_json::Value`. Match that arm and turn the payload into your type with `ReleaseNotes::deserialize(payload)` (a `&serde_json::Value` is a deserializer) or `serde_json::from_value(payload.clone())`. [Human in the loop](/agents/human-in-the-loop/) shows the whole loop. `StopWithArgs::default()` is the other form. Its schema has a single `output` string, and the payload is that string. `tools` keeps the default `stop`, so call `without_default_stop_tool` before `tools`, or add yours with `add_tool`. In this example, `stop_tool` is `StopWithArgs::with_parameters_schema(stop_schema())`: ```rust let mut builder = Agent::builder(); builder .llm(&openai) .system_prompt( "You are a workflow finisher. Summarize completed work and recommend next actions. \ When done, call the `stop` tool with the provided JSON schema. Always include \ `details`, using null when there is nothing to add. Never fabricate task status \ values outside the schema.", ) .without_default_stop_tool() .tools([stop_tool.clone()]) .on_stop(|_, reason, _| { Box::pin(async move { if let StopReason::RequestedByTool(_, payload) = reason && let Some(payload) = payload { println!( "agent stopped with structured payload:\n{}", to_string_pretty(&payload).unwrap_or_else(|_| payload.to_string()), ); } Ok(()) }) }); ``` The `on_stop` hook receives the same `StopReason`, so you can read the payload there instead of after `query` returns. The showcase’s output with `claude-sonnet-5-5`, through an OpenAI-compatible gateway, ends with the stop call and the deserialized notes: ```text → stop (version 0.34.0: highlights 2, breaking 1, fixes 1) stop ■ done ✔ RELEASE_NOTES.md 0.34.0 — This release adds the opt-in `swiftide-agents-contrib` crate and Perl support for tree-sitter. It also fixes duplicated text in OpenAI streaming and resolves security advisories, which removes AWS Console login from the Bedrock client. highlights (2) • Agent tools, hooks and executors in `swiftide-agents-contrib` [#1207] • Perl support for tree-sitter [#1198] breaking changes (1) • AWS Console login removed from the Bedrock client [#1215] fixes (1) • Repeated text in OpenAI streaming [#1216] ``` ## Failure reports `AgentCanFail` is a second control tool, named `task_failed`. The agent does not add it unless you pass it to `tools`. `AgentCanFail::default()` takes a `reason` string, and the agent stops with `StopReason::AgentFailed(Some(reason))`. `query` still returns `Ok`, so you check the stop reason. `AgentCanFail::with_parameters_schema` accepts your own schema: ```rust /// What the agent reports when it cannot finish. #[derive(Debug, Deserialize, JsonSchema)] #[schemars(deny_unknown_fields)] struct FailureReport { /// What went wrong, in one sentence. summary: String, /// What a person has to fix before the task can be retried. blocked_by: Vec, } let mut agent = Agent::builder() .llm(&openai) .tools([AgentCanFail::with_parameters_schema(schema_for!( FailureReport ))]) .build()?; agent.query("Restore last night's database backup.").await?; // With a custom schema the arguments arrive as one JSON-encoded string. if let Some(StopReason::AgentFailed(Some(Value::String(json)))) = agent.stop_reason() { let report: FailureReport = serde_json::from_str(json)?; println!("failed: {}", report.summary); for blocker in &report.blocked_by { println!(" blocked by: {blocker}"); } } ``` With a custom schema the tool serializes the arguments back to text. The payload is a `Value::String` that holds JSON, so you parse it a second time with `serde_json::from_str`. ## Stop reasons This run caps itself at 20 completions, lets one tool call fail twice, adds `task_failed`, and stops itself from a hook when the message history grows too long: ```rust let mut agent = Agent::builder() .llm(&openai) .system_prompt("Check that the release branch builds. Call `stop` when you are done.") .tools([AgentCanFail::default()]) .limit(20) .tool_retry_limit(2) .after_each(|agent| { Box::pin(async move { if agent.history().await?.len() > 200 { agent.stop("the history grew past 200 messages").await; } Ok(()) }) }) .build()?; let result = agent.query("Is the release branch green?").await; match agent.stop_reason() { Some(StopReason::RequestedByTool(_call, payload)) => println!("done: {payload:?}"), Some(StopReason::AgentFailed(reason)) => println!("the model gave up: {reason:?}"), Some(StopReason::ToolCallsOverLimit(call)) => println!("`{}` kept failing", call.name()), Some(StopReason::FeedbackRequired { tool_call, .. }) => { println!("`{}` waits for approval", tool_call.name()); } Some(StopReason::Error) => println!("the run failed"), Some(StopReason::NoNewMessages) => println!("turn ended without a tool call, or limit hit"), Some(StopReason::Other(reason)) => println!("stopped by our hook: {reason}"), Some(other) => println!("stopped: {other:?}"), None => println!("the agent did not stop; see the returned error"), } // `ToolCallsOverLimit` and `Error` also come back as an `Err`. result?; ``` `limit` caps the number of completions in one run. The counter starts at zero with every `query` or `run`. When it is reached, the agent logs a warning and stops with `StopReason::NoNewMessages`, and `query` returns `Ok`. No limit is set by default. `tool_retry_limit` caps how often the same tool call may fail before the agent stops with `StopReason::ToolCallsOverLimit` and `query` returns `AgentError::ToolError`. [Tools](/agents/tools/) explains how failures are counted. Each variant means something different for your code: * `RequestedByTool(tool_call, payload)`: a tool returned `ToolOutput::Stop`, usually a `stop` tool. `payload` is `None` for the default tool, the `output` value for `StopWithArgs::default()`, and the argument object for a custom schema. Treat it as success. * `AgentFailed(payload)`: a tool returned `ToolOutput::AgentFailed`, usually `task_failed`. Show the reason to a person, or retry with more context. * `ToolCallsOverLimit(tool_call)`: one call failed too often. `query` also returns the error. Fix the tool or its description, or raise `tool_retry_limit`. * `FeedbackRequired { tool_call, payload }`: a tool waits for approval. Record a `ToolFeedback` on the context and call `run`, as [Human in the loop](/agents/human-in-the-loop/) shows. * `Error`: a step failed, for example a completion request, and `query` returns the `AgentError`. * `NoNewMessages`: the model answered without a tool call, sent an empty reply, or the run reached `limit`. `query_once` and `run_once` also end with it after one step, unless a tool stopped the agent first. A new `query` continues the same message history. * `Other(text)`: your code stopped the agent. `agent.stop("text")` accepts any string. Hooks that receive `&mut Agent`, such as `after_each` and `after_tool`, can call it. `StopReason` is marked `#[non_exhaustive]`, so a `match` needs a wildcard arm. `agent.stop_reason()` returns `None` while the agent is `Pending` or `Running`. That includes a run that returned an error without recording a stop. Each variant also has an `is_*` method, and most have an `as_*` accessor such as `as_requested_by_tool`. ### Tool outputs `ToolOutput` has the same kind of accessors. `as_text` and `as_fail` return the text of a `Text` or `Fail` output; `as_stop` and `as_agent_failed` return the payload of a `Stop` or `AgentFailed` output. This `after_tool` hook reads them, and turns a failed `deploy` call into a failed run with `ToolOutput::agent_failed`: ```rust let agent = Agent::builder() .llm(openai) .after_tool(|_agent, tool_call, result| { if let Ok(output) = result { if let Some(text) = output.as_text() { println!("`{}` returned {} bytes", tool_call.name(), text.len()); } else if let Some(error) = output.as_fail().map(str::to_owned) { // A failed deploy ends the run instead of going back to the model. if tool_call.name() == "deploy" { *output = ToolOutput::agent_failed(json!({ "deploy": error })); } } else if let Some(payload) = output.as_stop() { println!("stopping with {payload}"); } else if let Some(reason) = output.as_agent_failed() { println!("giving up: {reason}"); } } Box::pin(async { Ok(()) }) }) .build()?; ``` The agent checks the output after the `after_tool` hooks run, so the rewritten output stops it with `StopReason::AgentFailed(Some(payload))`. `as_stop` and `as_agent_failed` return `None` when the output carries no payload, as with the default `stop` tool; match on `ToolOutput::Stop` or `ToolOutput::AgentFailed` to test the variant itself. ## Gotchas * To tell the cases of `NoNewMessages` apart, check whether the last assistant message has tool calls, or turn off the text-answer stop with `with_stop_on_assistant(false)` (see [Context and history](/agents/context-and-history/)). * `without_default_stop_tool` takes effect when you call `tools`, so call it first. * A `StopWithArgs` or `AgentCanFail` call whose arguments are not valid JSON is a tool error. It goes back to the model and counts toward `tool_retry_limit`. The default `Stop` ignores its arguments. * `to_string_pretty` on an `AgentCanFail` payload prints a quoted, escaped string; parse it with `serde_json::from_str` first. * Once the agent has stopped, `Agent::stop` does nothing. The first stop reason stays, and `on_stop` fires once. Tools the model called in the same completion as `stop` still run. * The payload is whatever the model sent. Handle a failed deserialization, as the showcase does. ## Reference * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`State`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.State.html) * [`Stop`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.Stop.html) * [`StopWithArgs`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.StopWithArgs.html) * [`AgentCanFail`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.AgentCanFail.html) * [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`ToolFeedback`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.ToolFeedback.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) ## Next [Human in the loop](/agents/human-in-the-loop/) # Streaming > Turn on streaming with on_stream, read the text and the tool call arguments in each stream chunk, and switch OpenAI-compatible clients to deltas only. You end up with an agent that prints the model’s text, and the arguments of its tool calls, while the model is still writing them. Streaming changes only how a completion arrives. The agent needs the `swiftide-agents` feature flag, and each model client needs its own: `openai`, `anthropic` or `aws-bedrock`. None of them is on by default. ```sh cargo add swiftide@0.34 --features swiftide-agents,openai ``` ## Streaming switch ```rust agents::Agent::builder() .llm(&openai) .on_stream(|_agent, response| { // We print the message chunk if it exists. Streamed responses also include // the full response (without tool calls) in `message` and an `id` to map them to // previous chunks for convenience. // // The agent uses the full assembled response at the end of the stream. if let Some(delta) = &response.delta { print!( "{}", delta .message_chunk .as_deref() .map(str::to_string) .unwrap_or_default() ); }; // If `stream_full` is disabled, response.message() will be the accumulated response // response.message() Box::pin(async move { Ok(()) }) }) // Every message added by the agent will be printed to stdout .on_new_message(move |_, msg| { let msg = msg.to_string(); Box::pin(async move { println!("\n---\nFinal message:\n {msg}"); Ok(()) }) }) .limit(5) .build()? .query("Why is the rust programming language so good?") .await?; ``` `openai` is an `OpenAI` client built with `stream_full(false)`; [Deltas only](#deltas-only) shows it. Registering `on_stream` turns streaming on. The agent then calls `complete_stream` on the model instead of `complete`, and runs the hook once for every stream chunk. Without a hook, `streaming(true)` on the builder streams as well. The hook receives `&Agent` and the chunk, a `ChatCompletionResponse`. The example prints `delta.message_chunk`, the text that arrived with this chunk. It never flushes stdout, so the text can arrive in bursts. The `on_new_message` hook prints every message the agent adds to the message history. For the assistant, that is the assembled message after the stream has ended. ## Stream chunks Each stream chunk is a whole `ChatCompletionResponse`. With the default settings it carries: * `id`, the same for every chunk of one completion. * `message`, the text accumulated so far. * `tool_calls`, `usage` and `reasoning`, the tool calls rebuilt, the token counts and the reasoning items so far. * `delta.message_chunk`, the text that arrived with this chunk. * `delta.tool_calls_chunk`, a map from a per-client index to a `ToolCallAccum` with `id`, `name` and `arguments`. `arguments` holds everything received so far, not only the newest fragment. The map stays on every later chunk of the completion, so remember what you already showed. The key differs by client. OpenAI chat completions use the tool call’s index, the Responses API the output item index. Anthropic and Bedrock use the content block index, which also counts text blocks. Match calls on `id` or `name`, not on the key. When the stream ends, the agent keeps the last chunk and uses it as the completion. It records that chunk’s message in the message history and runs its tool calls. A stream that yields nothing fails with `AgentError::EmptyStream`. A client without its own `complete_stream` sends one chunk with the whole response and no `delta`. ## Tool argument previews The release-notes-agent showcase streams its draft. The release notes are the `markdown` argument of the `write_changelog` tool, so they arrive as tool call arguments, not as text: examples/release-notes-agent/src/progress.rs ```rust /// Streams the model's text, and the release notes while they are being drafted. pub fn stream_draft(builder: &mut AgentBuilder) { let preview = Arc::new(Mutex::new(DraftPreview::default())); // Registering `on_stream` switches the agent to streaming completions. builder.on_stream(move |_agent, response| { let Some(delta) = &response.delta else { return Box::pin(async { Ok(()) }); }; let mut out = stdout().lock(); if let Some(text) = &delta.message_chunk { let _ = write!(out, "{text}"); } // The notes arrive as the (JSON) argument of `write_changelog`; show them as they // are written. let drafts = delta.tool_calls_chunk.iter().flat_map(HashMap::values); for call in drafts.filter(|call| call.name.as_deref() == Some(WRITE_CHANGELOG)) { if let (Some(id), Some(args)) = (&call.id, &call.arguments) { let mut preview = preview.lock().unwrap_or_else(PoisonError::into_inner); if let Some(text) = preview.advance(id, args) { let _ = write!(out, "{text}"); } } } let _ = out.flush(); Box::pin(async { Ok(()) }) }); } ``` The hook writes `message_chunk` to stdout. It then walks the entries of `tool_calls_chunk`, keeps the `write_changelog` call, and hands its id and accumulated arguments to `DraftPreview`. It flushes stdout after every chunk. [`DraftPreview`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/draft.rs) lives in `draft.rs`. Every chunk repeats the whole argument text, so the preview tracks how much it has shown. It decodes the complete part of the `markdown` value in the growing JSON (JavaScript Object Notation) text, stopping before an unfinished escape sequence. It returns only the text it has not returned before. A new call id starts a fresh preview. The hook is a `Fn` closure, so it cannot change what it captures, and it must be `Clone`. State that changes from chunk to chunk goes behind an `Arc>`, as `preview` does here. ## Captured stream These lines come from the showcase’s `OUTPUT.md`, a real run with `claude-sonnet-5-5` through an OpenAI-compatible gateway. `OUTPUT.md` cuts the draft after its first lines with `...`: ```text git_diff ✓ 17 lines ## 0.34.0 This release adds the opt-in `swiftide-agents-contrib` crate and Perl support for tree-sitter. It also fixes duplicated text in OpenAI streaming and resolves security advisories, which removes AWS Console login from the Bedrock client. ... → write_changelog <32 lines> write_changelog ⏸ waiting for approval ``` The draft appears before the `→ write_changelog` line. `on_stream` runs while the completion arrives, and the `before_tool` hook runs only after it is complete. After the approval, the agent runs the pending tool call without a new completion, so the draft streams once. ## Deltas only By default, the OpenAI-compatible clients clone the accumulated response, which grows with the answer, into every chunk. Turn that off with `stream_full(false)`: ```rust let openai = swiftide::integrations::openai::OpenAI::builder() .default_embed_model("text-embeddings-3-small") .default_prompt_model("gpt-4o-mini") // Only streams the delta, leave this out to stream the full response .stream_full(false) .build()?; ``` The last chunk is still the complete response, so the agent behaves the same. `delta.tool_calls_chunk` carries the same accumulated arguments with either setting, so the showcase’s preview works with the default too. [Deltas only](/models/overview/#deltas-only) lists the clients that have the setting and the fields each chunk drops. ## Gotchas * Before 0.34, the `GenericOpenAI` chat completions stream (OpenAI, Ollama, Groq, Gemini, OpenRouter and Dashscope) repeated the previous `message_chunk` on tool call fragments, the stop and usage frames, and the final chunk, so a hook that printed it printed text more than once. [#1216](https://github.com/bosun-ai/swiftide/pull/1216) fixed this. * The fix covers only the chat completions stream. The OpenAI Responses API (`use_responses_api(true)`, which the showcase’s `--responses-api` flag turns on), `Anthropic` and `AwsBedrock` still keep the previous `message_chunk` on tool call events. With them, both examples print the last text again for every tool call event. Their chunks carry the accumulated `message`, so printing the part of `message` beyond what you already printed avoids repeats. * The agent awaits the hook for every chunk before it reads the next one. A slow hook slows the stream. Hand slow work to a channel or a spawned task. * An error returned from `on_stream` is logged with `tracing` and does not stop the agent. * `on_stream` receives `&Agent`, so it cannot stop the agent or change its tools. Do that from `after_completion`. * Rust’s stdout is line buffered. `print!` without a newline may show nothing until a line ends or the buffer fills, so flush after each chunk. The first example does not flush; the showcase does. ## Reference * [`AgentBuilder::on_stream`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.on_stream) * [`OnStreamFn`](https://docs.rs/swiftide/0.34/swiftide/agents/hooks/trait.OnStreamFn.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatCompletionResponseDelta`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponseDelta.html) * [`ToolCallAccum`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCallAccum.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`GenericOpenAIBuilder::stream_full`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.stream_full) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) * [Feature flags](/reference/feature-flags/) ## Next [MCP tools](/agents/mcp/) # System prompts > Give an agent a system prompt as a Prompt, fill its Tera template from runtime values, change it before the first run, or leave it out. Here you build an agent whose system prompt is a Tera template, filled with values your program only knows at run time. The example is the system prompt of the [release notes agent](/showcases/release-notes-agent/). The code on this page needs the `swiftide-agents` feature flag, and the `openai` feature flag for the model: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents ``` ## Prompt as system message examples/release-notes-agent/src/agent.rs ```rust /// The system prompt is a Tera template; optional values are left undefined when absent. /// /// GitHub tools are only mentioned when the repository is on GitHub, since the prompt tells /// the agent which repository to query. fn system_prompt(&self, github_tools: bool) -> Prompt { let mut prompt = Prompt::from(SYSTEM_PROMPT) .with_context_value("since", self.since.as_str()) .with_context_value("github_tools", github_tools && self.github.is_some()); if let Some(version) = &self.next_version { prompt.insert_context_value("next_version", version.as_str()); } if let Some(github) = &self.github { prompt.insert_context_value("github", github.to_string()); } prompt } ``` The system prompt of an agent is a `Prompt`. `AgentBuilder::system_prompt` takes anything that converts into one, so a `&'static str`, a `String` or a finished `Prompt` all work. The showcase passes the result of this method to `system_prompt` in its `build` function. `SYSTEM_PROMPT` is [`src/system_prompt.md`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/system_prompt.md), included at compile time with `include_str!`. It is a Tera template, the Jinja-style template language Swiftide renders prompts with. Its last lines show how it handles optional values: ```text {% if github %} - Reference pull requests as Markdown links, exactly like `[#1207](https://github.com/{{ github }}/pull/1207)`. Reference a commit by its short hash only when it has no pull request.{% else %} - Reference pull requests as `#1207`, or commits by their short hash.{% endif %} {% if next_version %}- The version is `{{ next_version }}`.{% else %}- Choose the version yourself: before 1.0, bump the minor version of `{{ since }}` for new features or breaking changes, and the patch version for fixes only.{% endif %} ``` The template uses these constructs: * `{{ since }}` prints the tag the release starts from. Every run has one, so the method always sets it. * `{% if github_tools and github %}` adds the instructions for the GitHub tools only when the agent has them and the repository is on GitHub. * `{% if github %} ... {% else %} ... {% endif %}` picks the style of pull request references, and `{{ github }}` prints the repository as `owner/name`. * `{% if next_version %}` switches between “use this version” and “choose a version”, and `{{ next_version }}` prints it. * `{{ next_version | default(value="") }}` prints the version in the heading, or a placeholder when the user gave none. The optional values stay undefined when they are absent. Tera treats an undefined value in an `if` as false, and the `default` filter replaces it. That is why the method inserts `next_version` and `github` only when it has them. ## Template values `with_context_value` takes the prompt by value and returns it, so it chains onto `Prompt::from`. `insert_context_value` takes `&mut self`, which suits values you add conditionally, as the showcase does. Both accept anything that converts into a `tera::Value`: strings, numbers, booleans, and `serde_json::Value`. `with_context` merges a whole `tera::Context` at once. [Prompts and templates](/models/prompts-and-templates/#template-syntax) covers the template syntax, loops and the filters that are available. It also lists the `&str` and `format!` pitfalls. ## Values from a before_all hook Some values are only known when the agent starts. A `before_all` hook can read them and fill them in: ```rust let mut agent = Agent::builder() .llm(openai) .system_prompt("You review the changes on the branch `{{ branch }}`.") .before_all(|agent: &mut Agent| { Box::pin(async move { let branch = agent .context() .executor() .exec_cmd(&Command::shell("git branch --show-current")) .await? .into_string_lossy(); if let Some(prompt) = agent.system_prompt_mut() { prompt.insert_context_value("branch", branch.trim()); } Ok(()) }) }) .build()?; ``` The hook asks the tool executor for the current git branch and inserts it as `branch`. With the default local executor, the command runs in the working directory of your program. With a Docker executor, it runs in the container. On the first run, the `before_all` hooks run, then the agent renders the prompt and stores it in the context as a `ChatMessage::System`. A `before_all` hook is therefore the last place where a change still counts. `Agent::system_prompt_mut` returns `None` when the agent has no system prompt, so the hook only touches a prompt that exists. Before you call `build`, `AgentBuilder::system_prompt_mut` gives the same access on the builder. It also returns `None` when no prompt is set. A render error stops the run before the first completion. `query` returns `AgentError::FailedToRenderSystemPrompt`. The agent then stays pending, so the next `query` runs the `before_all` hooks and renders the prompt again. That gives a hook the chance to fill the missing value on a retry. ## Named templates A system prompt that several agents share can live in a named template: ```rust let mut templates = tera::Tera::default(); templates.add_raw_templates([ ( "release_rules", "Name the crate and the feature flag in every entry.", ), ( "release_manager", "You write the release notes for {{ project }}.\n\n{% include \"release_rules\" %}", ), ])?; Prompt::extend(&templates)?; let agent = Agent::builder() .llm(openai) .system_prompt( Prompt::from_compiled_template("release_manager") .with_context_value("project", "swiftide"), ) .build()?; ``` `Prompt::extend` merges your Tera instance into one template repository for the whole process. `Prompt::from_compiled_template` stores only the template name and looks it up when the agent renders the prompt. Named templates can include each other, so shared rules live in one partial, here `release_rules`. Swiftide does not re-export Tera. Add it to your own crate: ```sh cargo add tera@1.20 --no-default-features ``` ## Prompt removal `no_system_prompt` removes the system prompt from the builder, including one set earlier: ```rust let system_prompt = "You answer questions about the swiftide repository."; let mut first_agent = Agent::builder() .llm(openai) .system_prompt(system_prompt) .build()?; first_agent.query("Which crates does it have?").await?; // The stored history starts with the rendered system message. let history = first_agent.history().await?; let mut second_agent = Agent::builder() .llm(openai) .context( DefaultContext::default() .with_existing_messages(history) .await? .to_owned(), ) .system_prompt(system_prompt) // Removes the prompt set above, so the history keeps a single system message. .no_system_prompt() .build()?; second_agent.query("Which one holds the agents?").await?; ``` The second agent continues a stored message history that already holds a system message. Without `no_system_prompt`, its first run appends a second one. The call is useful when a shared builder function sets a prompt you do not want for a resumed run. [Context and history](/agents/context-and-history/) shows how the history is restored. ## Default prompt An agent has no system prompt unless you set one. Code written for 0.32 or earlier relied on a built-in default; [Upgrading](/reference/upgrading/#system-prompts) lists what to change. The model only sees what you write. A prompt for an agent with tools usually says what the task is, which tools to use in which order, and when to call `stop`. The release notes prompt does all three in its “How to work” list. ## Gotchas * A `Prompt::from` prompt without any context value is sent verbatim, without rendering. `{{ since }}` then reaches the model as literal text. Named templates always render. * An undefined value outside an `if` or the `default` filter fails rendering. Guard every optional value. * Changes to the system prompt after the first render do not reach the stored message. To change the instructions of a running conversation, replace the system message with `AgentContext::replace_history`, as [Context and history](/agents/context-and-history/) shows. * Setup errors after the render, such as a toolbox that fails to load, also leave the agent pending. The next `query` then adds the system message a second time. * Filters, functions and partials registered with `Prompt::extend` only work in named templates. A `Prompt::from` prompt renders with a fresh Tera instance. ## Reference * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) * [`AgentContext`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [Tera documentation](https://keats.github.io/tera/docs/) ## Next [Stopping and structured results](/agents/stopping-and-structured-results/) # Testing agents > Test Swiftide agents without a provider by scripting the model, calling tools directly, or matching exact requests with MockChatCompletion. You can test an agent end to end without provider credentials or network access: a scripted model decides which tools to call, and your test asserts on the tool outputs, the requests and the stop reason. `Agent`, `DefaultContext` and `StopReason` sit behind the `swiftide-agents` feature flag, which is off by default. The tests also need tokio for its test macros, serde_json and async-trait: ```sh cargo add swiftide@0.34 --features swiftide-agents cargo add --dev tokio --features macros,rt-multi-thread cargo add --dev serde_json async-trait anyhow ``` Swiftide re-exports serde_json, async_trait and anyhow under `swiftide::reexports`, so you can use those paths instead of adding the crates. ## Scripted models ```rust /// A model that answers with scripted responses and records every request it receives. #[derive(Clone, Default)] pub struct ScriptedLlm { replies: Arc>>, requests: Arc>>>, } impl ScriptedLlm { pub fn new(replies: impl IntoIterator) -> Self { Self { replies: Arc::new(Mutex::new(replies.into_iter().collect())), requests: Arc::default(), } } /// The messages of every request, in the order the agent sent them. pub fn requests(&self) -> Vec> { self.requests.lock().unwrap().clone() } } #[async_trait] impl ChatCompletion for ScriptedLlm { async fn complete( &self, request: &ChatCompletionRequest<'_>, ) -> Result { self.requests .lock() .unwrap() .push(request.messages().to_vec()); self.replies .lock() .unwrap() .pop_front() .ok_or_else(|| LanguageModelError::permanent("the script has no more replies")) } } /// A scripted reply that calls one tool with JSON arguments. pub fn reply_calling( name: &str, args: serde_json::Value, ) -> anyhow::Result { let call = ToolCall::builder() .id(format!("call_{name}")) .name(name) .args(args.to_string()) .build()?; ChatCompletionResponse::builder() .tool_calls(vec![call]) .build() } ``` An agent talks to its model through the [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) trait. Only `complete` is required. The default `complete_stream` wraps `complete` in a stream of one response, so a scripted model also works for an agent with `on_stream` hooks. `ScriptedLlm` hands out one prepared [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) per completion and records the messages of every request. When the script runs out, it returns a permanent `LanguageModelError`, and `query` fails with `AgentError::CompletionsFailed`. The state sits behind `Arc>` because `AgentBuilder::llm` takes a reference and stores a boxed clone. Your test keeps the original, and both share the same queue and request log. `reply_calling` builds a response that calls one tool. The arguments are a JSON (JavaScript Object Notation) string, as a provider would send them. ## Agent tests The tool under test is a plain function tool: ```rust /// Counts the words in a piece of text. /// /// # Arguments /// - `text`: The text to count the words of. #[swiftide::tool] pub async fn count_words(_context: &dyn AgentContext, text: &str) -> Result { Ok(text.split_whitespace().count().to_string().into()) } ``` The test scripts two turns: the model calls `count_words`, then the `stop` tool. ```rust #[tokio::test] async fn counts_the_words_then_stops() -> anyhow::Result<()> { let llm = ScriptedLlm::new([ reply_calling("count_words", json!({ "text": "one two three" }))?, reply_calling("stop", json!({}))?, ]); let mut agent = Agent::builder().llm(&llm).tools([count_words()]).build()?; agent .query("How many words are in `one two three`?") .await?; let history = agent.history().await?; let counted = history.iter().find_map(|message| match message { ChatMessage::ToolOutput(call, output) if call.name() == "count_words" => { output.as_text() } _ => None, }); assert_eq!(counted, Some("3")); assert!(matches!( agent.stop_reason(), Some(StopReason::RequestedByTool(call, None)) if call.name() == "stop" )); let requests = llm.requests(); assert_eq!(requests.len(), 2); assert!(matches!( requests[1].last(), Some(ChatMessage::ToolOutput(..)) )); Ok(()) } ``` The agent runs the real tool, records its output in the message history and sends the history back to the model. `stop` ends the run with `StopReason::RequestedByTool` and no payload. The test then checks three things: * the `ChatMessage::ToolOutput` for `count_words` holds `3` * the stop reason names the `stop` tool * the second request ended with the tool output, so the model saw the result Because the agent has no system prompt by default, the first request holds only the user message. To test a system prompt, set one on the builder and assert on the first `ChatMessage::System` in `llm.requests()`. A reply without tool calls, such as `ChatCompletionResponse::builder().message("Done.").build()`, scripts a turn where the model only answers in text. The default context stops the agent after such a turn, with `StopReason::NoNewMessages`. ## Direct tool invocation Most tool logic needs no agent at all. Call `invoke` on the tool with a context and a [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html): ```rust #[tokio::test] async fn count_words_rejects_a_misspelled_argument() -> anyhow::Result<()> { let tool = count_words(); let context = DefaultContext::default(); let call = ToolCall::builder() .id("call_1") .name("count_words") .args(json!({ "text": "one two" }).to_string()) .build()?; let output = tool.invoke(&context, &call).await?; assert_eq!(output.as_text(), Some("2")); let typo = ToolCall::builder() .id("call_2") .name("count_words") .args(json!({ "txt": "one two" }).to_string()) .build()?; let error = tool.invoke(&context, &typo).await.unwrap_err(); assert!(matches!(error, ToolError::WrongArguments(_))); Ok(()) } ``` [`DefaultContext::default()`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) runs commands through a `LocalExecutor` in the current directory, so tools that call `context.executor()` work too. Point it at a temporary directory with `DefaultContext::from_executor` when the tool touches files. Arguments that do not deserialize into the generated arguments struct return `ToolError::WrongArguments`. In the test above, the misspelled `txt` key leaves the required `text` argument missing. A call without arguments returns `ToolError::MissingArguments`. Extra keys are ignored: `deny_unknown_fields` only applies to the JSON schema the model sees. A misspelled key for an `Option` argument becomes `None` without an error. Inside an agent, both go back to the model as a failed tool output and count toward `tool_retry_limit`. ## MockChatCompletion For tests that pin down the exact requests, swiftide-core ships `MockChatCompletion` behind its `test-utils` feature flag. The swiftide crate has a `test-utils` feature too, but it does not re-export the module, so add swiftide-core as a dev dependency and import `swiftide_core::test_utils::MockChatCompletion`: ```sh cargo add --dev swiftide-core@0.34 --features test-utils ``` ```rust #[tokio::test] async fn sends_the_tool_output_back_to_the_model() -> anyhow::Result<()> { let llm = MockChatCompletion::new(); let question = ChatMessage::new_user("How many words are in `one two`?"); let count = ToolCall::builder() .id("call_1") .name("count_words") .args(json!({ "text": "one two" }).to_string()) .build()?; let stop = ToolCall::builder().id("call_2").name("stop").build()?; llm.expect_complete( ChatCompletionRequest::builder() .messages(vec![question.clone()]) .tools(Agent::default_tools()) .tool(count_words()) .build()?, Ok(ChatCompletionResponse::builder() .tool_calls(vec![count.clone()]) .build()?), ); llm.expect_complete( ChatCompletionRequest::builder() .messages(vec![ question, ChatMessage::new_assistant(None::, Some(vec![count.clone()])), ChatMessage::new_tool_output(count, ToolOutput::text("2")), ]) .tools(Agent::default_tools()) .tool(count_words()) .build()?, Ok(ChatCompletionResponse::builder() .tool_calls(vec![stop]) .build()?), ); let mut agent = Agent::builder().llm(&llm).tools([count_words()]).build()?; agent.query("How many words are in `one two`?").await?; assert!(agent.is_stopped()); Ok(()) } ``` Each `expect_complete` queues an expected [`ChatCompletionRequest`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionRequest.html) and the response to return. Expectations are consumed in the order you add them. A request that differs from the next expectation fails the test with both requests printed. An `Err` response reaches the agent as a permanent `LanguageModelError`. Request equality covers the messages and the full set of tool specs. The agent sends the spec of every tool it has, including the default `stop` tool, so the expected request lists `Agent::default_tools()` next to your own tools. When the last clone of the mock is dropped with expectations left over, it panics. A test that stops early fails even when every request it made matched. ## Showcase tests The release notes showcase has 27 tests. From the swiftide repository root: ```sh cargo test -p release-notes-agent ``` The session tests exercise the approval loop in `session.rs`: examples/release-notes-agent/src/session.rs ```rust /// Asks the agent for release notes and loops through review rounds until they are written or /// rejected. pub async fn draft_release_notes( agent: &mut Agent, task: Prompt, reviewer: &mut impl Reviewer, ) -> Result { agent.query(task).await?; let mut written = false; let mut nudged = false; loop { // `write_changelog` is wrapped in `ApprovalRequired`: the agent stops and waits. let tool_call = match agent.stop_reason() { Some(StopReason::FeedbackRequired { tool_call, .. }) => tool_call.clone(), // The custom `stop` tool carries the structured release notes. Some(StopReason::RequestedByTool(_, Some(notes))) if written => { let notes = ReleaseNotes::deserialize(notes) .context("the agent stopped with malformed release notes")?; return Ok(Outcome::Written { notes: Some(notes) }); } _ if written => return Ok(Outcome::Written { notes: None }), // A turn without a tool call, or the step limit. Some(StopReason::NoNewMessages) => { nudge_once(agent, &mut nudged).await?; continue; } other => bail!("the agent stopped without writing the notes: {other:?}"), }; let proposal: WriteChangelogArgs = serde_json::from_str(tool_call.args().unwrap_or_default())?; match reviewer.review(&proposal)? { Verdict::Approve => { agent .context() .feedback_received(&tool_call, &ToolFeedback::approved()) .await?; // Resumes the pending tool call, which now runs, and continues the loop. agent.run().await?; // The write itself can still fail; the agent then hears why and carries on. written |= succeeded(agent, &tool_call).await?; } Verdict::Revise(feedback) => { agent .context() .feedback_received(&tool_call, &ToolFeedback::refused()) .await?; // The refusal is recorded as the tool's output, followed by the feedback. agent.query(feedback).await?; } Verdict::Reject => return Ok(Outcome::Rejected), } } } ``` Five of the six session tests drive this loop with a scripted model like the one above. The showcase’s own `ScriptedLlm`, a simpler variant of the one above, takes `(tool name, arguments)` pairs, and an empty name scripts a turn without a tool call. A `ScriptedReviewer` plays the person who approves or asks for a revision, and those tests run against a temporary repository. They cover: * a revision round, then approval and the write * stopping without a write, which is an error * one nudge after a turn without a tool call, and an error after the second one * a draft that lists a pull request twice, which goes back to the model before review The duplicate check also ran against a live model. The other paths ran only in these tests. The sixth session test is a synchronous unit test. It builds a history by hand and checks that reaching the step limit is not mistaken for a turn without a tool call, although both report the same stop reason. The remaining tests are plain unit tests in the other modules. They cover system prompt rendering, decoding a streamed draft however it is chunked, and restarting the draft preview for a new tool call. They also cover finding a pull request listed in two entries, the stop tool schema, rendering the notes, tool names, clipping tool output, shell quoting, GitHub remote parsing, and rejecting unsafe revisions and paths. ## Gotchas * `swiftide_agents::test_utils` (`MockTool`, `MockHook`, `chat_request!`) is compiled only in test and debug builds, and its macros expand to `swiftide_core` paths. It serves swiftide’s own tests; do not build on it. * `MockChatCompletion::complete_stream` splits the response text on whitespace inside a spawned task. On a response without text that task panics, the stream closes empty, and `query` returns `AgentError::EmptyStream`. A streamed reply that only calls tools never delivers its tool calls. Script a message on every reply when the agent streams. * The agent rewrites JSON object arguments before it stores the assistant message, to drop duplicate keys. Build expected arguments with `serde_json`, as above, rather than as hand-typed strings with their own spacing. * The unit context `()` implements `AgentContext` for tools that never run commands. Its `executor()` panics, and it approves every feedback request. * A tool call that names a tool the agent does not have is skipped with a warning and gets no output. With a typo in a scripted tool name, the run ends with `StopReason::NoNewMessages` instead of the scripted stop. The next `query` or `run` on that agent fails with `AgentError::UnfulfilledToolCalls`. ## Reference * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`ChatCompletionRequest`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionRequest.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html) * [`ToolError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.ToolError.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`MockChatCompletion`](https://docs.rs/swiftide-core/0.34/swiftide_core/test_utils/struct.MockChatCompletion.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) ## Next [How tasks work](/tasks/overview/) # Tools > Write agent tools as async functions with #[swiftide::tool], as stateful structs with #[derive(Tool)], or by implementing Tool by hand with your own schema, and control how tool errors are retried. You end up with three kinds of tool: async functions the model can call, a tool with its own state, and a hand-written tool with a custom argument schema. The examples come from the release notes agent in `examples/release-notes-agent`, which drafts release notes from git history. Add the dependencies the macros expect: ```sh cargo add swiftide@0.34 --features swiftide-agents,openai cargo add serde --features derive cargo add schemars@1 async-trait ``` examples/release-notes-agent/src/tools.rs ```rust /// Lists every commit since a release tag, oldest first: hash, subject, body and the files it /// changed. /// /// # Arguments /// - `tag`: Tag to start from, e.g. `v0.33.0`. Use null for the latest tag. #[swiftide::tool] pub async fn git_log_since( context: &dyn AgentContext, tag: &Option, ) -> Result { let git = Git::new(context.executor().as_ref()); let since = match tag { Some(tag) => tag.parse()?, None => git.latest_tag().await?, }; let log = git.log_since(&since).await?; Ok(ToolOutput::text(clip( log, MAX_LOG_LINES, "Use git_diff on a specific path for details.", ))) } /// Shows the diff of one file or directory over a revision range. /// /// # Arguments /// - `range`: Revision range, e.g. `v0.33.0..HEAD`, or a single commit hash. /// - `path`: File or directory relative to the repository root. #[swiftide::tool] pub async fn git_diff( context: &dyn AgentContext, range: &str, path: &str, ) -> Result { let (range, path): (Revision, RepoPath) = (range.parse()?, path.parse()?); let diff = Git::new(context.executor().as_ref()) .diff(&range, &path) .await?; if diff.trim().is_empty() { return Ok(ToolOutput::text(format!( "No changes to {path} in {range}." ))); } Ok(ToolOutput::text(clip( diff, MAX_DIFF_LINES, "Diff a narrower path to see the rest.", ))) } /// Reads a file from the repository, e.g. to check a public API or the style of the changelog. /// /// # Arguments /// - `path`: File path relative to the repository root. #[swiftide::tool] pub async fn read_file(context: &dyn AgentContext, path: &str) -> Result { let path: RepoPath = path.parse()?; let content = context .executor() .exec_cmd(&Command::read_file(path.as_path())) .await?; Ok(ToolOutput::text(clip( content.into_string_lossy(), MAX_FILE_LINES, "The file is longer than shown.", ))) } ``` ## Function tools `#[swiftide::tool]` turns an async function into a tool. The first argument must be `&dyn AgentContext`. Every other argument becomes a tool argument that the model fills in. The macro generates three items from `git_log_since`: * a struct `GitLogSince` that implements `Tool`, * a function `git_log_since()` that returns it as a `Box`, ready for `AgentBuilder::tools`, * an argument struct `GitLogSinceArgs` that derives serde and `JsonSchema`. Its schema forbids extra properties, but deserialization ignores keys it does not know. The tool name the model sees is the function name, `git_log_since`. Since 0.33, the description is the doc comment text before the first heading. Each item under `# Arguments` describes one argument. A `# Parameters` heading works the same way. The tools parse the model’s strings into `Revision` and `RepoPath` before they call `context.executor()`, because the `LocalExecutor` in this example does not sandbox anything. See [Executors](/agents/executors/). A hook printed the first tool call of a real run with `claude-sonnet-5-5`: ```text Drafting release notes for the 7 commits since v0.33.0 with claude-sonnet-5-5 at http://127.0.0.1:8080/v1 (chat completions) → git_log_since v0.33.0 git_log_since ✓ 346 lines ``` ## Argument types An argument type must implement serde’s `Serialize` and `Deserialize`, `schemars::JsonSchema` and `Debug`, because the generated argument struct derives all four. The macro builds the JavaScript Object Notation (JSON) Schema of the arguments with `schemars::schema_for!`: ```rust /// An inclusive range of lines. #[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)] pub struct LineRange { /// First line, starting at 1 pub start: u32, /// Last line, inclusive pub end: u32, } /// Reads line ranges from files in the workspace. /// /// # Arguments /// - `paths`: Files to read, relative to the workspace root. /// - `range`: Optional line range to limit the output. Use null for the whole file. #[swiftide::tool] async fn read_lines( context: &dyn AgentContext, paths: Vec, range: Option, ) -> Result { let _ = (context, range); Ok(ToolOutput::text(paths.join("\n"))) } ``` * `&str`, `&[T]`, `&Vec` and `&Option` are deserialized as their owned types. Take any other type, such as `LineRange`, by value. * An `Option` argument is optional. The OpenAI integration marks every property as required, so there the model sends `null` instead of leaving the argument out. The `range` description tells the model so. * The doc comments of a struct argument become field descriptions. * A map type such as `HashMap` compiles, but the generated `tool_spec` panics when the agent builds its first request. Arguments that do not parse never reach your function: the model gets a `ToolError::WrongArguments` back. ## Stateful tools When a tool needs configuration, derive `Tool` on a struct: examples/release-notes-agent/src/tools.rs ```rust /// Prepends the approved release notes to the release notes file. /// /// The file is fixed by the CLI (`--output`), never chosen by the model. Wrap this tool in /// `ApprovalRequired` so a human signs off before anything is written. #[derive(Clone, Tool)] #[tool( description = "Prepends the finished release notes to the release notes file. A maintainer \ reviews the notes before they are written.", param( name = "markdown", description = "The complete release notes for this version, as GitHub flavored Markdown" ) )] pub struct WriteChangelog { file: PathBuf, } impl WriteChangelog { pub fn new(file: impl Into) -> Self { Self { file: file.into() } } async fn write_changelog( &self, context: &dyn AgentContext, markdown: &str, ) -> Result { let executor = context.executor(); // A missing file is started; any other failure must not overwrite the existing notes. let existing = if file_exists(executor.as_ref(), &self.file).await? { executor .exec_cmd(&Command::read_file(&self.file)) .await .map(CommandOutput::into_string_lossy)? } else { String::new() }; executor .exec_cmd(&Command::write_file( &self.file, prepend_notes(markdown, &existing), )) .await?; Ok(ToolOutput::text(format!("wrote {}", self.file.display()))) } } ``` The derive calls the method named after the struct in snake case, here `write_changelog`. The model never chooses the file: it comes from the struct. * The derive does not read doc comments. `description` is required, and each argument needs a `param`. * A `param` without a type is a `String`, so the method takes `&str`. * The struct must be `Clone`. Register the tool with `add_tool(WriteChangelog::new(path))`, or pass `WriteChangelog::new(path).into()` to `tools`. ## Hand-written tools Implement `Tool` yourself to write the tool spec by hand and parse the arguments in your own code. Describe the arguments as a type: ```rust #[derive(Clone)] struct WorkflowTool; #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[schemars( title = "WorkflowInstruction", description = "Choose a workflow action and optional payload", deny_unknown_fields )] struct WorkflowInstruction { #[schemars(description = "Which workflow action to execute")] action: WorkflowAction, #[schemars(description = "Optional payload forwarded to the workflow engine")] #[serde(default, skip_serializing_if = "Option::is_none")] payload: Option, } #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[serde(rename_all = "lowercase")] enum WorkflowAction { Start, Stop, Status, } ``` Then return that schema from `tool_spec`: ```rust #[swiftide::reexports::async_trait::async_trait] impl Tool for WorkflowTool { async fn invoke( &self, _agent_context: &dyn AgentContext, _tool_call: &ToolCall, ) -> Result { Ok(ToolOutput::text( "Workflow execution not implemented in this example", )) } fn name<'tool>(&'tool self) -> Cow<'tool, str> { Cow::Borrowed("workflow_tool") } fn tool_spec(&self) -> ToolSpec { ToolSpec::builder() .name("workflow_tool") .description("Executes a workflow action with strict input choices") .parameters_schema(workflow_schema()) .build() .expect("tool spec should be valid") } } fn workflow_schema() -> Schema { schema_for!(WorkflowInstruction) } ``` `invoke` receives the `ToolCall`, whose `args()` is the JSON string of the arguments. The agent drops duplicate keys and re-serializes the JSON first, so the string can differ from what the model sent. `name()` must match the `ToolSpec` name: the agent looks tools up by `name()`, the model calls the spec name. `ToolSpec::builder().build()` returns a `Result`. It checks the schema against Swiftide’s strict contract, so a `HashMap` field fails; see [Tool schema policies](/agents/providers-and-reasoning/#tool-schema-policies). The example calls `expect`, so an invalid schema panics at the first request. Cover it with a unit test that calls `tool_spec()`. A hand-written tool can wrap another one. `OneEntryPerChange` in the same `tools.rs` delegates to `WriteChangelog` and rejects drafts that list a change twice. ## Tool output A tool returns `Result`. `ToolOutput` is non-exhaustive and has these variants: * `Text(String)`: the result, added to the message history. `&str`, `String` and `CommandOutput` convert into it. * `Fail(String)`: a failure the model can fix. * `Stop(Option)`: stops the agent with `StopReason::RequestedByTool`. See [Stopping and structured results](/agents/stopping-and-structured-results/). * `FeedbackRequired(Option)`: stops the agent with `StopReason::FeedbackRequired` and leaves the tool call open. See [Human in the loop](/agents/human-in-the-loop/). * `AgentFailed(Option)`: stops the agent with `StopReason::AgentFailed`. ## Tool errors and retries `ToolError` has four variants: `WrongArguments`, `MissingArguments`, `ExecutionFailed` and `Unknown`. It converts from `serde_json::Error`, `CommandError` and `anyhow::Error`, so `?` works. ```rust /// Runs the tests of one crate in the workspace and returns their output. /// /// # Arguments /// - `package`: Name of the crate to test, e.g. `swiftide-core`. #[swiftide::tool] async fn run_tests(context: &dyn AgentContext, package: &str) -> Result { let is_crate_name = !package.is_empty() && package .chars() .all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_'); if !is_crate_name { return Ok(ToolOutput::fail(format!( "`{package}` is not a crate name. Use a package name from Cargo.toml." ))); } let command = Command::shell(format!("cargo test --package {package}")); match context.executor().exec_cmd(&command).await { Ok(output) => Ok(output.into()), Err(CommandError::NonZeroExit(output)) => { Ok(ToolOutput::text(format!("Tests failed:\n{output}"))) } Err(error) => Err(ToolError::execution_failed(error)), } } ``` A bad crate name returns `Ok(ToolOutput::fail(..))`, and the model tries again. An executor error returns `Err`: the agent records it as a `Fail` output and counts the failure. `tool_retry_limit` sets how many failures of the same call the agent feeds back to the model: ```rust let mut agent = Agent::builder() .llm(openai) .tools([run_tests()]) .tool_retry_limit(2) .build()?; if let Err(error) = agent.query("Run the tests of swiftide-core.").await { if let Some(StopReason::ToolCallsOverLimit(call)) = agent.stop_reason() { eprintln!("gave up on `{}`: {error}", call.name()); } } ``` With a limit of 2, the third identical failure stops the agent. The full rules for `tool_retry_limit` and `limit` are on [Stopping and structured results](/agents/stopping-and-structured-results/#stop-reasons). ## Attribute descriptions Instead of doc comments you can pass `description` and `param` attributes. An explicit attribute wins over the doc comment: ```rust #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] struct FormatTimestampRequest { /// Prefix to prepend to the timestamp. prefix: String, /// Timestamp to format. timestamp: String, } #[swiftide::tool(description = "Get the current UTC date and time in RFC3339 format")] async fn current_utc_time(context: &dyn AgentContext) -> Result { let command_output = context .executor() .exec_cmd(&Command::shell("date -u +\"%Y-%m-%dT%H:%M:%SZ\"")) .await?; Ok(command_output.into()) } #[swiftide::tool( description = "Format a timestamp with a caller-provided prefix", param(name = "request", description = "Timestamp formatting input") )] async fn format_timestamp( _context: &dyn AgentContext, request: FormatTimestampRequest, ) -> Result { Ok(ToolOutput::text(format!( "{}{}", request.prefix, request.timestamp ))) } ``` `description` also accepts a path to a `const`, such as `description = READ_FILE`. ## Gotchas * The generated code refers to `::swiftide::` and `::serde` paths. The crate that uses the macros needs `swiftide` with the default `macros` feature flag and `serde` with `derive`. * Every argument after the context must be listed under `# Arguments` or as a `param`. A doc bullet with an empty description is dropped, and a missing argument fails to compile with “parameters are missing from the spec”. * Tools are unique by name. `add_tool` replaces a tool with the same name, and `tools` replaces every tool added before it. To bring your own `stop`, see [Stopping and structured results](/agents/stopping-and-structured-results/). * The tool calls of one completion run concurrently on separate Tokio tasks, so two calls to a tool that writes the same file can race. ## Reference * [`#[swiftide::tool]`](https://docs.rs/swiftide/0.34/swiftide/attr.tool.html) and [`#[derive(Tool)]`](https://docs.rs/swiftide/0.34/swiftide/derive.Tool.html) * [`Tool`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/trait.Tool.html) * [`ToolSpec`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolSpec.html) and [`ToolSpecBuilder`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolSpecBuilder.html) * [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`ToolError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.ToolError.html) * [`AgentContext`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html) * [`AgentBuilder::tool_retry_limit`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.tool_retry_limit) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) ## Next [Context and message history](/agents/context-and-history/) # Caching > Make re-indexing incremental with a Redb, Redis or DuckDB node cache that skips files whose path and content have not changed. Add a node cache to your indexing pipeline and a second run skips every file whose path and content have not changed, before any large language model (LLM) call or embedding is made. ## Redb cache The [ask-your-codebase](/showcases/ask-your-codebase/) showcase keeps its node cache in an embedded Redb file next to the code it indexes: examples/ask-your-codebase/src/codebase.rs ```rust /// Remembers which files were fully indexed, so unchanged files are skipped next time. pub fn node_cache(&self) -> Result { std::fs::create_dir_all(&self.data_dir)?; Redb::builder() .database_path(self.data_dir.join("cache.redb")) .table_name("indexed_files") .build() } /// Embedded vector store with one vector per chunk and the source path next to it. pub fn vector_store(&self) -> Result { let uri = self.lancedb_dir(); LanceDB::builder() .uri(uri.to_str().context("Index path is not valid UTF-8")?) .table_name("code_chunks") .vector_size(embedding::DIMENSIONS) .with_vector(EmbeddedField::Combined) .with_metadata(PATH_FIELD) .build() } ``` `Redb::builder()` takes a `database_path` and creates the file if it does not exist. It does not create missing parent directories, so the showcase calls `create_dir_all` first. You can pass an open `redb::Database` with `database` instead of a path. `table_name` defaults to `swiftide`. The second function builds the LanceDB storage, covered on [Storage](/indexing/storage/). The pipeline adds the cache with `filter_cached`: examples/ask-your-codebase/src/index.rs ```rust let stats = indexing::Pipeline::from_loader( FileLoader::new(codebase.root()).with_extensions(&languages.extensions()), ) .filter(test_code::not_test_files(codebase.clone())) // Drop unit tests before the cache, which keys on path and content. Changing the content // after it would make the cache miss on every run. .then(test_code::without_unit_tests) .then(files_found.count()) // Skip files whose path and content are unchanged since the last successful run .filter_cached(codebase.node_cache()?) .then(files_changed.count()) .then(record_relative_path(codebase.clone())) // Split along syntax boundaries (functions, impls, ...) with tree-sitter .then_chunk(ChunkByLanguage::try_new(languages, CHUNK_SIZE)?) // Ask the LLM which questions each chunk answers; stored as metadata and embedded .then( MetadataQACode::from_client(llm) .prompt_template(QUESTIONS_PROMPT) .num_questions(QUESTIONS_PER_CHUNK) .concurrency(concurrent_requests) .build()?, ) // Embed chunk + metadata locally, in batches .then_in_batch(Embed::new(embedding::local_model()?).with_batch_size(embedding::BATCH_SIZE)) .then_store_with(codebase.vector_store()?) .run() .await?; ``` `filter_cached` drops every node the cache knows. Every other node continues to the chunker, the LLM and the embedder. The first run indexes the `swiftide-agents` crate from scratch, with `claude-haiku-5-5` writing the questions and FastEmbed computing the embeddings, as described in [Output](/indexing/overview/#output): ```text $ ./target/debug/ask-your-codebase index swiftide-agents --model claude-haiku-5-5 Indexing Rust files in swiftide-agents ... Indexed 34 chunks from 14 new or changed files (14 files found, 0 unchanged) in 91.9s ``` The second run, with no file changed: ```text $ ./target/debug/ask-your-codebase index swiftide-agents --model claude-haiku-5-5 Indexing Rust files in swiftide-agents ... Indexed 0 chunks from 0 new or changed files (14 files found, 14 unchanged) in 0.6s ``` The cache skips all 14 files, so no prompts are sent and nothing is embedded. ## Cache keys The cache key is a node id: a UUID (universally unique identifier) computed from the node’s path and chunk, not its metadata. The built-in chunkers set `parent_id` on every chunk to the id of the node that entered the pipeline, the first ancestor, even across several chunkers. Caches use `parent_id` when it is set. One key therefore covers a file and all of its chunks. Each backend puts a prefix in front of the id: | Backend | Key format | Prefix setting | | ------- | ----------------- | ------------------------------------------------------------------------------------------------------- | | Redb | `.` | `cache_key_prefix`, empty by default | | Redis | `:` | second argument of `Redis::try_from_url`, or `cache_key_prefix` on `Redis::builder()`, empty by default | | DuckDB | `.` | `cache_key_prefix`, empty by default | The pipeline records the id a node has when it passes `filter_cached`. At the end it marks the `parent_id` of each finished node, or its own id when `parent_id` is not set. The file is cached only when the two ids match. Three things break the match silently. A step after `filter_cached` that rewrites `chunk` or `path` in place changes the node’s id. The file is then never marked and is processed again on every run, with or without a chunker. Keep content-changing steps in front of `filter_cached`, like the showcase does with `test_code::without_unit_tests`. Steps that only add metadata are safe after it. A custom chunker must keep the parent. The built-in chunkers build chunks with `Node::into_chunks`, which sets `parent_id`. A closure or `ChunkerTransformer` that creates fresh nodes with `TextNode::new` or `Node::builder` leaves `parent_id` empty, so the file is never marked. Derive chunks with `into_chunks` or `Node::build_from_other`. The path is part of the id. The same file loaded through `./src` and through an absolute path has two keys. The showcase canonicalizes the codebase root for this reason. ## When nodes are cached Since 0.33, `run` marks a file in the cache only after the stream has completed and every chunk of that file reached the end. A file with a failed chunk is left out and retried on the next run. Two consequences follow: * If `run` returns an error, nothing from that run is cached. The same holds when the process stops before `run` returns. * A source whose chunker produces zero chunks never reaches the end of the stream. It is never cached, and every step up to the chunker runs for it again on each run. To keep progress when some nodes fail, log and drop the errors so that `run` completes. This snippet needs the `redis`, `qdrant` and `openai` feature flags: ```rust // Change the prefix, for example to "docs-v2", to index every file again let cache = Redis::try_from_url(redis_url, "docs-v1")?; Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .filter_cached(cache) .then_chunk(ChunkMarkdown::from_chunk_range(100..1500)) .then(MetadataQAText::new(openai.clone())) // Log and drop failed chunks so the run completes. Their files stay uncached and are // retried next run; every other file is cached. .log_errors() .filter_errors() .then_in_batch(Embed::new(openai).with_batch_size(32)) .then_store_with( Qdrant::builder() .vector_size(1536) .collection_name("docs") .build()?, ) .run() .await?; ``` The file behind a failed chunk stays out of the cache and is retried next run. Every other file is cached. [Errors](/indexing/streams-and-concurrency/#errors) covers where to place `log_errors` and `filter_errors`. ## Backends | Backend | Feature flag | Node type | Notes | | ----------------------------------------------------------------------------------------- | ------------ | -------------- | ------------------------------------------------------- | | [`Redb`](https://docs.rs/swiftide/0.34/swiftide/integrations/redb/struct.Redb.html) | `redb` | `TextNode` | Embedded file, no service to run | | [`Redis`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.Redis.html) | `redis` | any chunk type | Shared between machines; also a storage | | [`Duckdb`](https://docs.rs/swiftide/0.34/swiftide/integrations/duckdb/struct.Duckdb.html) | `duckdb` | any chunk type | Cache table `swiftide_cache` by default; also a storage | Turn on the feature flag for the backend you pick: ```sh cargo add swiftide@0.34 --features redb ``` `Redis::try_from_url` returns a cache for `String` chunks. For another chunk type, use `Redis::try_build_from_url` or `Redis::builder()`. With Redis, the cache is one line. This example also needs the `qdrant`, `openai` and `tree-sitter` feature flags: examples/index_codebase.rs ```rust indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["rs"])) .filter_cached(Redis::try_from_url(redis_url, "swiftide-examples")?) .then(MetadataQACode::new(openai_client.clone())) .then_chunk(ChunkCode::try_for_language_and_chunk_size( "rust", 10..2048, )?) .then_in_batch(Embed::new(openai_client.clone()).with_batch_size(10)) .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-examples") .build()?, ) .run() .await?; ``` `MetadataQACode` runs before chunking here. That is safe, because it only adds metadata and the id does not change. No backend fails the pipeline on a cache error. `get` logs the error and reports the node as not cached, so an unreachable Redis means every node is processed again. ## Custom cache To write your own cache, implement [`NodeCache`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.NodeCache.html). This one keeps the ids in memory: ```rust use std::{ collections::HashSet, sync::{Arc, Mutex}, }; use async_trait::async_trait; use swiftide::{indexing::TextNode, traits::NodeCache}; use uuid::Uuid; /// Remembers indexed sources in memory, for tests or one-off runs. #[derive(Debug, Clone, Default)] pub struct InMemoryCache { indexed: Arc>>, } /// The same key the pipeline marks: the source id, shared by all chunks of a file. fn source_id(node: &TextNode) -> Uuid { node.parent_id().unwrap_or_else(|| node.id()) } #[async_trait] impl NodeCache for InMemoryCache { type Input = String; async fn get(&self, node: &TextNode) -> bool { match self.indexed.lock() { Ok(indexed) => indexed.contains(&source_id(node)), Err(error) => { tracing::error!("node cache lock poisoned: {error}"); false } } } async fn set(&self, node: &TextNode) { self.set_by_id(source_id(node)).await; } async fn set_by_id(&self, id: Uuid) { match self.indexed.lock() { Ok(mut indexed) => { indexed.insert(id); } Err(error) => tracing::error!("node cache lock poisoned: {error}"), } } } ``` Implement it with `#[async_trait]`, declare `type Input` as the chunk type, and make the type `Send`, `Sync`, `Debug` and `Clone`. `get` returns `bool`, and `set` and `set_by_id` return nothing, so log errors instead of returning them. Key on `parent_id`, falling back to `id()`, so that `get`, `set` and `set_by_id` agree. The pipeline calls `set_by_id` with the source id when it marks a file. It is a required method since 0.33, so existing custom caches must add it. `clear` is optional, and its default implementation panics. ## Cache invalidation `NodeCache::clear` empties a cache, but each backend does it differently: * Redb deletes its table. * DuckDB drops its cache table and does not recreate it. Every later lookup on the same `Duckdb` logs an error and misses, so build a new `Duckdb` before the next run. * Redis sends `DEL` with the pattern `*`. Redis treats that as a literal key name, so the call removes nothing. With an empty prefix, `clear` returns an error instead. Change the prefix, for example from `docs-v1` to `docs-v2` in the partial-progress snippet. A new prefix works on every backend. Use it to index everything again, for example after you change a prompt or the embedding model. Clearing the cache does not touch the storage. A changed file’s new chunks get new ids and are stored next to the old ones, which stay in LanceDB until you delete the `.ask-your-codebase/` directory. ## Reference * [`NodeCache`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.NodeCache.html) * [`Pipeline::filter_cached`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.filter_cached) * [`Pipeline::log_errors`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.log_errors) * [`Pipeline::filter_errors`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.filter_errors) * [`Node`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Node.html) * [`Redb`](https://docs.rs/swiftide/0.34/swiftide/integrations/redb/struct.Redb.html) * [`Redis`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.Redis.html) * [`Duckdb`](https://docs.rs/swiftide/0.34/swiftide/integrations/duckdb/struct.Duckdb.html) * [Feature flags](/reference/feature-flags/) * [ask-your-codebase source](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase) ## Next [Storage](/indexing/storage/) # Chunking > Split source code with tree-sitter and markdown or plain text by characters, write a chunker as a closure, and keep path and metadata on every chunk. After this page your indexing pipeline splits code along syntax boundaries, markdown and text by characters, and any other format with a closure, with chunks sized for your embedder. A chunker takes one node and returns many; you add it with `then_chunk`. ## Source code `ChunkCode` needs the `tree-sitter` [feature flag](/reference/feature-flags/): ```sh cargo add swiftide@0.34 --features tree-sitter ``` ```rust // At most 1500 bytes per chunk, the default let python = ChunkCode::try_for_language("python")?; // At most 2000 bytes per chunk; chunks of 200 bytes or less are dropped let rust = ChunkCode::try_for_language_and_chunk_size(SupportedLanguages::Rust, 200..2000)? .with_concurrency(4); Pipeline::from_loader(FileLoader::new("src").with_extensions(&["rs"])) .then_chunk(rust) .then_store_with(MemoryStorage::default()) .run() .await?; ``` `ChunkCode` parses each file with tree-sitter and splits it along syntax boundaries such as functions, `impl` blocks and classes. It adds top-level items to a chunk. It closes the chunk when the next item would bring it to the maximum or past it. An item that is too large on its own is split along its children. The size is a `ChunkSize`, and it counts bytes. A number such as `1500` is a maximum. A range such as `200..2000` adds a minimum: a chunk of 200 bytes or less is dropped. `try_for_language` uses a maximum of 1500 bytes. The language is a `SupportedLanguages` value or a name such as `"python"`; an unknown name returns an error. `with_concurrency` sets how many files are chunked at once. Without it the chunker uses the pipeline’s concurrency. When tree-sitter finds a syntax error, `ChunkCode` logs a warning and returns the whole file as one chunk, whatever its size. ## Several languages `ChunkCode` handles one language. The ask-your-codebase showcase indexes several in one pipeline with its own chunker: examples/ask-your-codebase/src/languages.rs ```rust /// Splits each file with a tree-sitter `ChunkCode` for the file's language. /// /// `ChunkCode` is built for a single language; this chunker picks the right one per file from /// its extension, so one pipeline can index a polyglot codebase. #[derive(Clone)] pub struct ChunkByLanguage { chunkers: Vec<(SupportedLanguages, ChunkCode)>, } impl ChunkByLanguage { /// Chunks are between `chunk_size.start` and `chunk_size.end` bytes; smaller ones are dropped. pub fn try_new(languages: &SourceLanguages, chunk_size: Range) -> Result { let chunkers = languages .0 .iter() .map(|&language| { let chunker = ChunkCode::try_for_language_and_chunk_size(language, chunk_size.clone())?; Ok((language, chunker)) }) .collect::>()?; Ok(Self { chunkers }) } fn chunker_for(&self, path: &Path) -> Option<&ChunkCode> { let extension = path.extension()?.to_str()?; self.chunkers .iter() .find(|(language, _)| language.file_extensions().contains(&extension)) .map(|(_, chunker)| chunker) } } #[async_trait] impl ChunkerTransformer for ChunkByLanguage { type Input = String; type Output = String; async fn transform_node(&self, node: TextNode) -> IndexingStream { match self.chunker_for(&node.path) { Some(chunker) => chunker.transform_node(node).await, None => anyhow!("No chunker for {}", node.path.display()).into(), } } } ``` The showcase builds it with a range of `10..4096` bytes. The [tree-sitter integration](/integrations/tree-sitter/#several-languages-in-one-pipeline) explains how it picks a language when extensions overlap. A file without a matching chunker becomes an error in the stream, and the first error stops `run`. Load only the extensions you have chunkers for, as the showcase does, or add `log_errors()` and `filter_errors()` ([Streams and concurrency](/indexing/streams-and-concurrency/)). ## Markdown and text ```rust // Aims for 100 characters, at most 1499; chunks under 100 bytes after trimming are dropped let markdown = ChunkMarkdown::from_chunk_range(100..1500); // At most 1000 characters; only blank chunks are dropped let text = ChunkText::from_max_characters(1000); ``` `ChunkMarkdown` and `ChunkText` use the text-splitter crate and count characters, not bytes. `ChunkMarkdown` prefers to split at headings, then at paragraphs and other blocks. `ChunkText` splits at blank lines and line breaks, then at sentences and words. Both default to a maximum of 2056 characters. With `from_chunk_range`, text-splitter treats the start of the range as the size to aim for and the end, exclusive, as a hard limit. A chunk grows past the start only when that keeps a larger unit, such as a section, in one piece. Chunks that are empty after trimming, or shorter than the start, are dropped. That check uses the byte length of the trimmed text, which equals the character count only for ASCII. The [docs-qa showcase](/showcases/docs-qa/) splits documentation pages with `ChunkMarkdown::from_chunk_range(120..1500)`, because shorter chunks were mostly badges and stray headings. ## Closures ```rust let paragraphs = |node: TextNode| { let chunks: Vec = node .chunk .split("\n\n") .map(str::trim) .filter(|paragraph| !paragraph.is_empty()) .map(str::to_owned) .collect(); // Each chunk keeps the path, metadata and embed mode of `node` IndexingStream::iter(node.into_chunks(chunks).map(Ok)) }; Pipeline::from_loader(FileLoader::new("notes").with_extensions(&["txt"])) .then_chunk(paragraphs) .then_store_with(MemoryStorage::default()) .run() .await?; ``` A closure `Fn(TextNode) -> IndexingStream` that is `Clone + Send + Sync` is a chunker. It works on text nodes only. `IndexingStream::iter` turns an iterator of `Result` into a stream. `node.into_chunks` builds one node per chunk from the input node. Every chunk keeps the path, metadata, vectors and embed mode of that node. For anything more involved, or for other node types, implement `ChunkerTransformer` on a struct as the showcase does. ## Node expansion ```rust // A JSON Lines file holds one record per line let records = |node: TextNode| { let lines: Vec = node .chunk .lines() .filter(|line| !line.trim().is_empty()) .map(str::to_owned) .collect(); IndexingStream::iter( node.into_chunks(lines) .enumerate() .map(|(index, mut record)| { record.metadata.insert("Record", index); Ok(record) }), ) }; Pipeline::from_loader(FileLoader::new("exports").with_extensions(&["jsonl"])) .then_expand(records) .then_chunk(ChunkText::from_max_characters(1000)) .then_store_with(MemoryStorage::default()) .run() .await?; ``` `then_expand` runs the same `ChunkerTransformer` as `then_chunk`; only its tracing span is named differently. Use it when one node holds several records, as in this JSON Lines file, and use `then_chunk` when you cut content to size. ## Metadata and parents Metadata that a node carries before chunking is copied to every chunk. The showcase stores each file’s relative path as metadata before it chunks, so every chunk can cite its file. The same holds for metadata written by a large language model (LLM). Put a transformer such as `MetadataQACode` before the chunker and every chunk of a file carries the questions for the whole file. Put it after the chunker for metadata per chunk, at one request per chunk. `MetadataQACode` also needs the `tree-sitter` feature flag. Since 0.33 every chunk also carries `parent_id`, the id its source node had when it reached the first chunker. The node cache marks that id when the file’s chunks finish. Two consequences matter while you chunk: * A step between `filter_cached` and the chunker that changes the path or the chunk breaks that match, so the file is never cached. Edit the content before `filter_cached`, as the showcase does. * A file whose chunks all fall under the minimum is never cached and is processed again on every run. [Cache keys](/indexing/caching/#cache-keys) explains the match in full. ## Gotchas * Size chunks for the embedder. With the default embed mode the metadata is embedded together with the chunk, so leave room for it under the model’s input limit. * Chunks come out of `then_chunk` in no particular order, and chunks of different files interleave. `ChunkCode` sets `offset` on each chunk, a running byte count over the chunks before it in the same file. The text chunkers keep the offset of the input node. * Only `ChunkCode` sees syntax. Running `ChunkText` over code cuts through functions at line or sentence boundaries. * `ChunkCode` cannot split a syntax node that has no children. A single token larger than the maximum, such as a long comment or string, is silently left out of the chunks. ## Reference * [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkCode.html) * [`ChunkSize`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/enum.ChunkSize.html) * [`SupportedLanguages`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/enum.SupportedLanguages.html) * [`ChunkMarkdown`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkMarkdown.html) * [`ChunkText`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkText.html) * [`ChunkerTransformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChunkerTransformer.html) * [`Pipeline::then_chunk`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.then_chunk) and [`Pipeline::then_expand`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.then_expand) * [`Pipeline::filter_errors`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.filter_errors) * [`Node::into_chunks`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Node.html#method.into_chunks) * [`IndexingStream`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.IndexingStream.html) * The [ask-your-codebase](/showcases/ask-your-codebase/) and [docs-qa](/showcases/docs-qa/) showcases, the [tree-sitter integration](/integrations/tree-sitter/) and [feature flags](/reference/feature-flags/) ## Next [Transformers and metadata](/indexing/transformers-and-metadata/) # Code-aware indexing > Index a codebase with tree-sitter so each code chunk carries a compressed outline of its file, questions about the code, and the symbols it defines and uses. This page builds an indexing pipeline that splits source code along syntax boundaries and gives every chunk the context of its file: an outline trimmed to what the chunk uses, questions the code answers, and the symbols it defines and uses. Tree-sitter is a parser library with one grammar per language. Swiftide wraps it in a chunker and four transformers behind the `tree-sitter` feature flag. The example below also uses OpenAI, a Redis node cache and Qdrant storage: ```sh cargo add swiftide@0.34 --features tree-sitter,openai,redis,qdrant ``` ## Reduced context pipeline This example indexes a Rust codebase. It sets up an OpenAI client, the Redis URL for the node cache and a chunk size. It needs `OPENAI_API_KEY`, plus Redis and Qdrant running locally: ```rust let openai_client = integrations::openai::OpenAI::builder() .default_embed_model("text-embedding-3-small") .default_prompt_model("gpt-3.5-turbo") .build()?; let redis_url = std::env::var("REDIS_URL") .as_deref() .unwrap_or("redis://localhost:6379") .to_owned(); let chunk_size = 2048; ``` The pipeline gives every chunk an outline of its file, trimmed to what that chunk uses: ```rust indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["rs"])) .filter_cached(Redis::try_from_url( redis_url, "swiftide-examples-codebase-reduced-context", )?) .then( indexing::transformers::OutlineCodeTreeSitter::try_for_language( "rust", Some(chunk_size), )?, ) .then(MetadataQACode::new(openai_client.clone())) .then_chunk(ChunkCode::try_for_language_and_chunk_size( "rust", 10..chunk_size, )?) .then(indexing::transformers::CompressCodeOutline::new( openai_client.clone(), )) .then_in_batch(Embed::new(openai_client.clone()).with_batch_size(10)) .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-examples-codebase-reduced-context") .build()?, ) .run() .await?; ``` The steps run in this order: 1. `filter_cached` skips files Redis has already seen, keyed on path and content as [Cache keys](/indexing/caching/#cache-keys) explains. 2. `OutlineCodeTreeSitter` parses the whole file and stores an outline in the `Outline` metadata field. An outline is the file with the bodies of functions and methods removed. Signatures, types, imports and comments stay. The second argument skips files smaller than that many bytes. Here it equals the chunk size, because a file that fits in one chunk needs no outline. 3. `MetadataQACode` asks a large language model (LLM) for questions and answers about the file. Its default prompt includes the outline when the node has one. 4. `ChunkCode` splits the file into chunks of more than 10 and up to 2048 bytes. 5. `CompressCodeOutline` asks the LLM to keep only the outline lines the chunk needs and writes the result back to `Outline`. 6. `Embed` embeds the chunks in batches of 10, and Qdrant stores them with vectors of 1536 dimensions, the size of `text-embedding-3-small`. See [Caching](/indexing/caching/) and [Storage](/indexing/storage/) for the node cache and the storage. Chunking copies the metadata of a file to every chunk, so each chunk starts with the full outline and the questions of its file. The default embed mode embeds metadata together with the chunk. Compressing the outline keeps that embedded context small and specific. ## Cost per chunk `MetadataQACode` runs before chunking here, so it sends one prompt per file and every chunk shares the answers. Move it after `then_chunk` to get questions per chunk, at one prompt per chunk. `CompressCodeOutline` sends one prompt per chunk of a file larger than the outline threshold. Chunks of smaller files have no outline and cost nothing. On a large codebase that is still many requests. Lower the concurrency or wrap the client in `LanguageModelWithBackOff` to stay within rate limits. ## Question prompt `MetadataQACode` writes its reply to `Questions and Answers (code)`. The default prompt asks generic questions. The [ask-your-codebase](/showcases/ask-your-codebase/) showcase replaces it with a prompt about behaviour, which matches the questions developers ask: examples/ask-your-codebase/src/index.rs ````rust /// Asks for the questions a developer would ask that a chunk answers. /// /// The default `MetadataQACode` prompt asks generic questions ("What does this code do?"). /// Questions about behaviour, phrased the way a developer would ask them, match real questions /// much better. const QUESTIONS_PROMPT: &str = r" # Task List the questions a developer could ask about this codebase that the code below answers, each with a short answer. They are stored with the code to help a search find it. Focus on behaviour: when and in which order things happen, what starts or ends a process, which conditions and configuration change the outcome, what happens on errors, and which extension points (traits, hooks, builders, options) exist and when they apply. # Constraints - Write {{ questions }} questions and answers. - Phrase questions the way a developer would ask them, e.g. 'When does the agent stop?'. - Name the relevant types, functions and variants in the answers. - Only use what can be derived from the code. - Respond only in this format: Q1: A1: # Code ``` {{ node.chunk }} ``` "; ```` The prompt receives `questions` and the node. The default prompt also renders `{{ outline }}`, which `MetadataQACode` sets when the node has an `Outline` field. The showcase prompt leaves it out, because the showcase builds no outline. Add `{{ outline }}` to your own prompt when you use it in the pipeline above, or the outline does not reach the LLM. [Tree-sitter](/integrations/tree-sitter/#questions-per-chunk) shows how to pass a prompt template and the number of questions. ## References and definitions `MetadataRefsDefsCode` adds the symbols each chunk defines and uses, without an LLM: ```rust indexing::Pipeline::from_loader(FileLoader::new("src").with_extensions(&["rs"])) .then_chunk(ChunkCode::try_for_language_and_chunk_size( "rust", 10..2048, )?) .then(MetadataRefsDefsCode::try_from_language("rust")?) .then_store_with(MemoryStorage::default()) .run() .await?; // The same transformer on one node let refs_defs = MetadataRefsDefsCode::try_from_language("rust")?; let node = refs_defs .transform_node(TextNode::new(r#"fn main() { println!("Hello"); }"#)) .await?; assert_eq!(node.metadata.get(NAME_REFERENCES).unwrap(), "println"); assert_eq!(node.metadata.get(NAME_DEFINITIONS).unwrap(), "main"); ``` It stores the symbols the chunk defines in `Definitions (code)` and the symbols it uses but does not define in `References (code)`. Both lists are sorted, without duplicates, and joined with commas. A field is left out when nothing is found. The constants `NAME_DEFINITIONS` and `NAME_REFERENCES` in `swiftide::indexing::transformers::metadata_refs_defs_code` hold the field names. Run it after chunking, so each chunk lists its own symbols. ## Languages `ChunkCode`, `OutlineCodeTreeSitter` and `MetadataRefsDefsCode` take a language. `MetadataQACode` and `CompressCodeOutline` take only an LLM client. The constructor for references and definitions is `try_from_language`; the others use `try_for_language`. [Tree-sitter](/integrations/tree-sitter/#supported-languages) lists which languages each transformer supports. Perl needs the `tree-sitter-perl` feature flag. ## Code splitter outside a pipeline `CodeSplitter` is the splitter inside `ChunkCode`. Use it directly when you need chunks outside an indexing pipeline, for example in a tool: ```rust // Case-insensitive; "c#", "c-sharp", "cpp" and "c++" parse too let language: SupportedLanguages = "python".parse()?; assert_eq!(language.file_extensions(), ["py"]); // Drop chunks of 100 bytes or less, split anything above 1500 bytes let splitter = CodeSplitter::builder() .try_language(language)? .chunk_size(100..1500) .build()?; let chunks = splitter.split(source)?; // Code with a syntax error comes back whole, as one chunk let broken = splitter.split("def main(:\n pass")?; assert_eq!(broken.len(), 1); ``` `SupportedLanguages` parses names without regard to case, and alternative names such as `"c#"`, `"c-sharp"`, `"cpp"` and `"c++"` work. Parsing an unknown name with `parse()` returns a strum parse error that says `Matching variant not found`. The constructors and `try_language` return the error `Treesitter language not supported` instead. The chunk size is a `ChunkSize`. A number is a maximum in bytes; `ChunkSize::default()` is 1500 bytes. A range sets a minimum and a maximum, and a chunk is kept only when it is longer than the minimum. `CodeSplitter::new(language)` uses the default size. ## Gotchas * The splitter drops chunks at or below the minimum and syntax leaves larger than the maximum, and returns a file with a syntax error as one chunk. [Chunking](/indexing/chunking/) covers these cases. * `OutlineCodeTreeSitter` returns an error for a file with a syntax error. The first error aborts `run`, so add `log_errors()` and `filter_errors()` when your codebase contains files that do not parse. See [Streams and concurrency](/indexing/streams-and-concurrency/). * `OutlineCodeTreeSitter` and `MetadataRefsDefsCode` panic on code in a language they do not handle. In a pipeline the panic happens inside a spawned task, so `run` fails with a join error. * Chunk sizes for code are bytes. `ChunkMarkdown` and `ChunkText` count characters, see [Markdown and text](/indexing/chunking/#markdown-and-text). ## Reference * [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkCode.html) * [`OutlineCodeTreeSitter`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.OutlineCodeTreeSitter.html) * [`CompressCodeOutline`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.CompressCodeOutline.html) * [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQACode.html) * [`MetadataRefsDefsCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataRefsDefsCode.html) and the [`metadata_refs_defs_code`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/metadata_refs_defs_code/index.html) module with `NAME_DEFINITIONS` and `NAME_REFERENCES` * [`CodeSplitter`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/struct.CodeSplitter.html) * [`ChunkSize`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/enum.ChunkSize.html) * [`SupportedLanguages`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/enum.SupportedLanguages.html) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`TextNode`](https://docs.rs/swiftide/0.34/swiftide/indexing/type.TextNode.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [Tree-sitter integration](/integrations/tree-sitter/) and [feature flags](/reference/feature-flags/) ## Next [How querying works](/querying/overview/) # Embedding > Add dense and sparse embeddings to nodes with Embed and SparseEmbed, choose the embedded text with EmbedMode, and store each embedded field as a named vector in Qdrant. Here every chunk in your indexing pipeline ends up with a dense and a sparse vector, stored in a Qdrant collection that a retriever can search. You add an embedder as a batch transformer, choose which text it embeds, and configure the storage to keep the vectors you need. Add the `fastembed` and `qdrant` feature flags: ```sh cargo add swiftide@0.34 --features fastembed,qdrant ``` ## Dense and sparse embeddings The docs-qa showcase gives every chunk a dense and a sparse embedding: examples/docs-qa/src/index.rs ```rust indexing::Pipeline::from_loader(loader) .then_chunk(ChunkMarkdown::from_chunk_range( MIN_CHUNK_CHARS..MAX_CHUNK_CHARS, )) // A short title and keywords per chunk; both end up in the embedded text and help the // sparse (keyword) side of hybrid search in particular. .then( MetadataTitle::from_client(llm.clone()) .prompt_template(TITLE_PROMPT) .build()?, ) .then( MetadataKeywords::from_client(llm.clone()) .prompt_template(KEYWORDS_PROMPT) .build()?, ) .then_in_batch(Embed::new(embedders.dense.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_in_batch(SparseEmbed::new(embedders.sparse.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_store_with(qdrant.clone()) .run() .await ``` `Embed::new` takes any `EmbeddingModel` and fills the node’s `vectors`. `SparseEmbed::new` takes any `SparseEmbeddingModel` and fills `sparse_vectors`. A dense vector captures what the text means. A sparse vector weights the words in it, which helps keyword-like matches in hybrid search. Both are batch transformers, so you add them with `then_in_batch`. Each batch becomes one model call. `with_batch_size` sets the nodes per batch, with a pipeline default of 256. `with_concurrency` sets how many batches run at the same time. Both work on text nodes (`TextNode`) only. A second `Embed` step replaces the dense vectors of the first one instead of adding to them. ## Embed mode A node’s embed mode decides which texts the embedder sees, each under an embedded field: | `EmbedMode` | Embedded fields | | ------------------------------ | ----------------------------------------------------- | | `SingleWithMetadata` (default) | `Combined` | | `PerField` | `Chunk`, plus one `Metadata(name)` per metadata field | | `Both` | all of the above | `Combined` is the metadata as `key: value` lines, a newline, then the chunk. Metadata keys come in sorted order. `Chunk` is the chunk alone. `Metadata(name)` is the value of one metadata field. Values that are not strings are embedded as JSON (JavaScript Object Notation). `Pipeline::with_embed_mode` sets the mode on every node that passes that point. Chunkers that use `Node::into_chunks` copy the mode to each chunk, so you can set it right after the loader. Qdrant names each vector after its field: `Combined`, `Chunk` or `Metadata: Title`. Sparse vectors add `_sparse`, as in `Combined_sparse`. LanceDB prefixes the column with `vector_`, as in `vector_combined`. ## Named vectors With `EmbedMode::PerField`, every node carries several dense vectors. Qdrant stores each one as a named vector: examples/store_multiple_vectors.rs ```rust indexing::Pipeline::from_loader(FileLoader::new("LICENSE")) .with_concurrency(1) .with_embed_mode(EmbedMode::PerField) .then_chunk(ChunkMarkdown::from_chunk_range(20..2048)) .then(MetadataQAText::new(openai_client.clone())) .then(MetadataSummary::new(openai_client.clone())) .then(MetadataTitle::new(openai_client.clone())) .then(MetadataKeywords::new(openai_client.clone())) .then_in_batch(Embed::new(openai_client.clone()).with_batch_size(10)) .log_all() .filter_errors() .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-multi-vectors") .with_vector(EmbeddedField::Chunk) .with_vector(EmbeddedField::Metadata(metadata_qa_text::NAME.into())) .with_vector(EmbeddedField::Metadata(metadata_summary::NAME.into())) .with_vector( VectorConfig::builder() .embedded_field(EmbeddedField::Metadata(metadata_title::NAME.into())) .distance(Distance::Manhattan) .build()?, ) .with_vector(EmbeddedField::Metadata(metadata_keywords::NAME.into())) .build()?, ) .run() .await?; ``` The example embeds the chunk and four metadata fields that a large language model (LLM) writes. Each `with_vector` call adds a named vector for one embedded field. Each metadata transformer module has a `NAME` constant, such as `metadata_title::NAME`, so the vector names match. `VectorConfig::builder()` overrides the distance or the size of one vector. When a node has several vectors, Qdrant stores only the fields you configured and skips the rest. Without `with_vector`, the collection has one vector for `Combined`. The first `with_vector` call replaces that default. [Similarity search](/integrations/qdrant/#similarity-search) on a collection with named vectors always queries `Combined`, which the example above lacks. To search it, use `EmbedMode::Both` and add `with_vector(EmbeddedField::Combined)`. ### Single vector collections With exactly one dense vector and no sparse vector, Qdrant uses one unnamed vector, both in the collection and in each node that has a single vector. That node skips the field filter. So `with_vector(EmbeddedField::Chunk)` alone breaks in two ways: * `PerField` or `Both` nodes with several vectors arrive as named vectors, and Qdrant rejects the write. * Nodes in the default mode get their `Combined` vector stored without an error. With a single configured vector, configure the field your embed mode produces. ## Hybrid collections Hybrid search queries a dense and a sparse vector. `HybridSearch` uses `Combined` for both by default, and `with_dense_vector_field` and `with_sparse_vector_field` change each one on its own. The hybrid search example loads both models and configures both vectors for `Combined`: examples/hybrid_search.rs ```rust // Ensure all batching is consistent let batch_size = 64; let fastembed_sparse = FastEmbed::try_default_sparse().unwrap().to_owned(); let fastembed = FastEmbed::try_default().unwrap().to_owned(); // Set up openai with the mini model, which is great for indexing let openai = openai::OpenAI::builder() .default_prompt_model("gpt-4o-mini") .build() .unwrap(); // Set up qdrant and use the combined fields (metadata + chunks) for both sparse and dense // vectors let qdrant = Qdrant::builder() .batch_size(batch_size) .vector_size(384) .with_vector(EmbeddedField::Combined) .with_sparse_vector(EmbeddedField::Combined) .collection_name("swiftide-hybrid-example") .build()?; ``` `FastEmbed::try_default()` loads BGE (BAAI General Embedding) small, a dense model with 384 dimensions. `FastEmbed::try_default_sparse()` loads a SPLADE (Sparse Lexical and Expansion) model for sparse vectors. `vector_size` must match the dense model, here 384. Qdrant stores a sparse vector only when its field also has a dense vector configured. So `with_sparse_vector(EmbeddedField::Chunk)` without `with_vector(EmbeddedField::Chunk)` adds a sparse vector that never gets filled. ## Embedding models [FastEmbed](/integrations/fastembed/) runs models on your central processing unit (CPU). It is the only integration that produces sparse embeddings. One instance holds one model, and passing a sparse instance to `Embed` fails with “Expected dense model, got sparse”. [FastEmbed](/integrations/fastembed/#model-choice) shows how to load another model, such as a code-trained one. Provider embeddings come from a client with an embedding model. The named vectors example uses OpenAI, which needs the `openai` feature flag: examples/store_multiple_vectors.rs ```rust let openai_client = integrations::openai::OpenAI::builder() .default_embed_model("text-embedding-3-small") .default_prompt_model("gpt-4o") .build()?; ``` Without `default_embed_model`, every `embed` call fails with “Model not set”. The example sets `vector_size(1536)` to match `text-embedding-3-small`. [Models](/models/overview/) lists which client implements `EmbeddingModel`. Index and query with the same model. A new model needs a new index. ## Batch sizes Batching happens at three levels: * `with_batch_size` on `Embed` sets the nodes per model call, and `batch_size` on `FastEmbed` the texts per model run, as [FastEmbed](/integrations/fastembed/#batch-size) explains. * `batch_size` on the storage sets the nodes per write. Qdrant defaults to 50. [Batching](/indexing/storage/#batching) covers how the storage receives them. With `PerField`, a node with four metadata fields gives five texts, so a batch of 10 nodes sends 50 texts in one call. ## Gotchas * A failed model call turns the whole batch into one error. `filter_errors` then drops every node of that batch, see [Errors before a batch step](/indexing/streams-and-concurrency/#errors-before-a-batch-step). * `LanguageModelWithBackOff` does not retry `embed` calls, as [errors and backoff](/models/errors-backoff-and-usage/) explains. * `Combined` puts the metadata before the chunk. A model that truncates long input cuts the end of the chunk first. FastEmbed truncates at its token limit without an error. `fastembed::TextInitOptions::with_max_length` raises that limit, as ask-your-codebase does. * `PerField` embeds every metadata field, including the ones your storage skips. Each one costs embedding time. * A chunker that builds nodes with `TextNode::new` instead of `Node::into_chunks` resets the embed mode to the default. * Clones of a `FastEmbed` share one model behind a lock, so a higher `with_concurrency` does not speed it up. ## Reference * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.SparseEmbed.html) * [`EmbedMode`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbedMode.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [`Pipeline::then_in_batch`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.then_in_batch) * [`Pipeline::with_embed_mode`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.with_embed_mode) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`SparseEmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SparseEmbeddingModel.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`QdrantBuilder::with_vector`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.QdrantBuilder.html#method.with_vector) * [`QdrantBuilder::with_sparse_vector`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.QdrantBuilder.html#method.with_sparse_vector) * [`VectorConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.VectorConfig.html) * [`HybridSearch`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.HybridSearch.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.OpenAI.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`Node::into_chunks`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Node.html#method.into_chunks) * [Feature flags](/reference/feature-flags/) ## Next [Caching](/indexing/caching/) # Loading data > Start an indexing pipeline from files on disk, nodes you build in code, a tool executor, web pages, Kafka, Fluvio, Parquet or a custom Loader. Every indexing pipeline starts with a stream of nodes. You fill that stream from files, web pages, Kafka, Fluvio, Parquet or your own data, with a loader and `Pipeline::from_loader` or with any stream and `Pipeline::from_stream`. ## Files ```rust let languages = [SupportedLanguages::Rust, SupportedLanguages::Python]; let extensions: Vec<&str> = languages .iter() .flat_map(|language| language.file_extensions().iter().copied()) .collect(); // Extensions without the leading dot; each call adds to the list let loader = FileLoader::new("src") .with_extensions(&extensions) .with_extensions(&["md"]); Pipeline::from_loader(loader) .then_chunk(ChunkText::from_max_characters(2000)) .then_store_with(MemoryStorage::default()) .run() .await?; ``` `FileLoader::new` takes a directory or a single file. Its walk uses the `ignore` crate’s defaults. It skips hidden files and files excluded by `.ignore` files, and by `.gitignore` rules when the root is inside a git repository. `with_extensions` keeps only files with those extensions. Without any call the loader yields every file. `SupportedLanguages::file_extensions` returns the extensions tree-sitter knows for a language. It needs the `tree-sitter` feature flag. `FileLoader` itself needs none. Each file becomes one `TextNode`: * `chunk` holds the file content, with invalid UTF-8 replaced. * `path` is relative when the root is relative. The node id hashes path and chunk, so changing the root from relative to absolute misses the [node cache](/indexing/caching/). * A file that cannot be read becomes an error in the stream. The first error ends `run` unless you filter errors, see [Errors](/indexing/streams-and-concurrency/#errors). ## Nodes in memory ```rust let mut faq = TextNode::new("A loader turns a data source into a stream of nodes."); faq.path = "faq/loading.md".into(); faq.metadata.insert("Source", "faq"); let nodes = vec![ faq, TextNode::new("An indexing pipeline does nothing until you call run."), ]; let storage = MemoryStorage::default(); Pipeline::from_stream(nodes) .then_store_with(storage.clone()) .run() .await?; println!("Stored {} nodes", storage.get_all_values().await.len()); ``` `Pipeline::from_stream` takes anything that converts into an `IndexingStream`. That includes a `Vec`, a boxed `Stream` of results and a Tokio `mpsc` receiver. A channel lets another task produce nodes while the pipeline runs: ```rust let (sender, receiver) = tokio::sync::mpsc::channel(100); tokio::spawn(async move { for line in ["first message", "second message"] { if sender.send(Ok(TextNode::new(line))).await.is_err() { break; } } // Dropping the sender ends the stream, and with it the run }); Pipeline::from_stream(receiver) .then_store_with(MemoryStorage::default()) .run() .await?; ``` The stream ends when every sender is dropped. Until then, `run` keeps waiting for nodes. ## Files from a tool executor ```rust let executor = LocalExecutor::new("."); let files = executor .stream_files(Path::new("src"), Some(vec!["rs".to_string()])) .await?; Pipeline::from_stream(files) .then_chunk(ChunkText::from_max_characters(2000)) .then_store_with(MemoryStorage::default()) .run() .await?; ``` `ToolExecutor::stream_files` returns the files below a path as an `IndexingStream`. `LocalExecutor` uses `path` as given and does not resolve it against the executor’s working directory. `LocalExecutor` needs the `swiftide-agents` feature flag. ## Web pages examples/scraping_index_to_markdown.rs ```rust indexing::Pipeline::from_loader(ScrapingLoader::from_spider( Website::new("https://www.bosun.ai/") .with_limit(1) .to_owned(), )) .then(HtmlToMarkdownTransformer::default()) .then_chunk(ChunkMarkdown::from_chunk_range(20..2048)) .log_all() .then_store_with(MemoryStorage::default()) .run() .await?; ``` `ScrapingLoader` crawls with the `spider` crate and needs the `scraping` feature flag. `ScrapingLoader::from_spider` takes a configured `Website`, here limited to one page. To configure it, add `spider` at the version swiftide uses (2.52), or the `Website` types do not match. Each node carries the raw HTML of one page, with the page URL as its `path`. `HtmlToMarkdownTransformer` converts the HTML to markdown and drops `script` and `style` tags. ### Spider Cloud Since 0.33, `ScrapingLoader` crawls through Spider Cloud when `SPIDER_CLOUD_API_KEY` is set and not empty. It reads these environment variables once per process: | Variable | Effect | | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | | `SPIDER_CLOUD_API_KEY` | Turns Spider Cloud on | | `SPIDER_CLOUD_MODE` | `proxy`, `api`, `unblocker` or `fallback`. Anything else selects smart mode, which uses the unblocker only against bot protection. | | `SPIDER_CLOUD_API_URL` | Optional API endpoint | Spider Cloud cannot reach hosts that only your machine can route to. A start URL on `localhost`, a `.localhost` or `.local` name, a loopback address, or a private or link-local IPv4 address is crawled directly. ## Kafka and Fluvio examples/kafka.rs ```rust static LOADER_TOPIC: &str = "loader"; static STORAGE_TOPIC: &str = "storage"; let mut client_config = ClientConfig::new(); client_config.set("bootstrap.servers", "localhost:9092"); client_config.set("group.id", "group_id"); client_config.set("auto.offset.reset", "earliest"); let loader = Kafka::builder() .client_config(client_config.clone()) .topic(LOADER_TOPIC) .build() .unwrap(); let storage = Kafka::builder() .client_config(client_config) .topic(STORAGE_TOPIC) .create_topic_if_not_exists(true) .batch_size(2usize) .build() .unwrap(); ``` examples/kafka.rs ```rust indexing::Pipeline::from_loader(loader) .then_in_batch(Embed::new(FastEmbed::try_default().unwrap()).with_batch_size(10)) .then_store_with(storage) .run() .await?; ``` The pipelines embed with `FastEmbed`, which needs the `fastembed` feature flag. `Kafka` needs the `kafka` feature flag, which builds librdkafka and needs cmake and a C toolchain. It takes an `rdkafka` `ClientConfig`, re-exported as `integrations::kafka::ClientConfig`. As a loader it subscribes to `topic` and turns each message payload into a node. Messages without a UTF-8 payload are skipped. The node metadata gets `kafka_topic`, `kafka_partition` and `kafka_offset`, plus `kafka_timestamp` and `kafka_key` when the message has them. `Kafka` is also a storage, here writing to a second topic. examples/fluvio.rs ```rust static TOPIC_NAME: &str = "hello-rust"; static PARTITION_NUM: u32 = 0; let loader = Fluvio::builder() .consumer_config_ext( ConsumerConfigExt::builder() .topic(TOPIC_NAME) .partition(PARTITION_NUM) .offset_start(fluvio::Offset::from_end(1)) .build() .unwrap(), ) .build() .unwrap(); ``` `Fluvio` needs the `fluvio` feature flag and is a loader only. It takes a `ConsumerConfigExt` with the topic, partition and start offset. `Offset` comes from the `fluvio` crate, which you add at swiftide’s version (0.50). Each record becomes a node with its key in the `fluvio_key` metadata field, which is null when the record has no key. A Kafka pipeline waits for new messages, so `run` does not return on its own. For both loaders, a receive error ends the run. ## Parquet ```rust let loader = Parquet::builder() .path("data.parquet") .column_name("chunk") .build()?; Pipeline::from_loader(loader) .then_chunk(ChunkText::from_max_characters(2000)) .then_store_with(MemoryStorage::default()) .run() .await?; ``` `Parquet` needs the `parquet` feature flag. `batch_size` sets the rows read per batch and defaults to 1024. The loader reads one column and turns every non-null value into a node. The column must hold strings. A column of another type yields an error, and a missing column panics. Batches are flattened unordered, so node order can differ from row order. Each load prints the column descriptors to stderr. ## Custom loaders When no loader fits, implement `Loader`. The [docs-qa](/showcases/docs-qa/) showcase loads markdown and `llms-full.txt` sources as one node per page: examples/docs-qa/src/loader.rs ```rust /// Yields one [`TextNode`] per documentation page, with the page title as metadata. #[derive(Debug, Clone)] pub struct DocsLoader { sources: Vec, } impl DocsLoader { pub fn new(sources: Vec) -> Self { Self { sources } } } impl Loader for DocsLoader { type Output = String; fn into_stream(self) -> IndexingStream { stream::iter(self.sources) .then(DocsSource::pages) .map_ok(|pages| stream::iter(pages).map(|page| Ok(page_node(page)))) .try_flatten() .boxed() .into() } // Only needed when the loader is used as a `Box`. fn into_stream_boxed(self: Box) -> IndexingStream { self.into_stream() } } fn page_node(page: Page) -> TextNode { let mut node = TextNode::new(page.markdown); node.path = page.origin.into(); node.metadata.insert(PAGE_TITLE, page.title.as_str()); node } ``` * `type Output` is the chunk type of the nodes. Built-in transformers expect `String`. * `into_stream` is synchronous and consumes the loader. Put slow work inside the stream, like the page downloads this loader starts through `DocsSource::pages`, so it runs when the pipeline polls it. * A failing source becomes an `Err` item, not a panic. * `into_stream_boxed` is only needed when you use the loader as a `Box`. Its default implementation panics. * `Loader` requires `Send`, `Sync` and `DynClone`, which any `Clone` type implements. ## Gotchas * `Pipeline::from_loader` calls `into_stream` right away, so work starts when you build the pipeline, not when you call `run`. `Kafka` subscribes to its topic. `Fluvio` connects. `Parquet` opens the file. `ScrapingLoader` starts crawling, and panics outside a Tokio runtime. * `Fluvio` and `Parquet` use `tokio::task::block_in_place`, which panics on a current-thread runtime. `#[tokio::main]` uses a multi-thread runtime by default; `#[tokio::test]` does not, so use `#[tokio::test(flavor = "multi_thread")]`. * `Kafka`, `Fluvio` and `Parquet` panic when they cannot create their consumer or open their file. ## Feature flags Install the feature flags you need, for example: ```sh cargo add swiftide@0.34 --features tree-sitter,scraping,kafka,fluvio,parquet,fastembed ``` [Feature flags](/reference/feature-flags/) lists them all. ## Reference * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`Loader`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Loader.html) * [`IndexingStream`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.IndexingStream.html) * [`Pipeline::from_loader`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.from_loader) * [`Pipeline::from_stream`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.from_stream) * [`TextNode`](https://docs.rs/swiftide/0.34/swiftide/indexing/type.TextNode.html) * [`SupportedLanguages`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/enum.SupportedLanguages.html) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) * [`ScrapingLoader`](https://docs.rs/swiftide/0.34/swiftide/integrations/scraping/struct.ScrapingLoader.html) * [`HtmlToMarkdownTransformer`](https://docs.rs/swiftide/0.34/swiftide/integrations/scraping/struct.HtmlToMarkdownTransformer.html) * [`Kafka`](https://docs.rs/swiftide/0.34/swiftide/integrations/kafka/struct.Kafka.html) * [`Fluvio`](https://docs.rs/swiftide/0.34/swiftide/integrations/fluvio/struct.Fluvio.html) * [`Parquet`](https://docs.rs/swiftide/0.34/swiftide/integrations/parquet/struct.Parquet.html) ## Next [Chunking](/indexing/chunking/) # How indexing works > An indexing pipeline streams nodes from a loader through filters, transformers, chunkers and embedders into storage, and the pipeline does its work only when you await run(). You can read a Swiftide indexing pipeline line by line and know when each step runs. The example comes from the [ask-your-codebase](/showcases/ask-your-codebase/) showcase. It prepares the source files of a codebase for retrieval-augmented generation (RAG): it splits them into chunks, describes each chunk with a large language model (LLM), embeds it, and stores it in LanceDB. ## The ask-your-codebase pipeline examples/ask-your-codebase/src/index.rs ```rust let stats = indexing::Pipeline::from_loader( FileLoader::new(codebase.root()).with_extensions(&languages.extensions()), ) .filter(test_code::not_test_files(codebase.clone())) // Drop unit tests before the cache, which keys on path and content. Changing the content // after it would make the cache miss on every run. .then(test_code::without_unit_tests) .then(files_found.count()) // Skip files whose path and content are unchanged since the last successful run .filter_cached(codebase.node_cache()?) .then(files_changed.count()) .then(record_relative_path(codebase.clone())) // Split along syntax boundaries (functions, impls, ...) with tree-sitter .then_chunk(ChunkByLanguage::try_new(languages, CHUNK_SIZE)?) // Ask the LLM which questions each chunk answers; stored as metadata and embedded .then( MetadataQACode::from_client(llm) .prompt_template(QUESTIONS_PROMPT) .num_questions(QUESTIONS_PER_CHUNK) .concurrency(concurrent_requests) .build()?, ) // Embed chunk + metadata locally, in batches .then_in_batch(Embed::new(embedding::local_model()?).with_batch_size(embedding::BATCH_SIZE)) .then_store_with(codebase.vector_store()?) .run() .await?; ``` ![Indexing pipeline](/diagrams/diagram-pipeline.svg) Each node passes through the steps from top to bottom until a step drops it: 1. `FileLoader` walks the codebase root and emits one node per file with a matching extension. 2. `filter` drops test files. The closure sees each `Result`, so errors pass through and still surface. 3. `then(test_code::without_unit_tests)` cuts unit tests out of Rust files. 4. `filter_cached` drops files whose path and content did not change since the last successful run. `codebase.node_cache()` is a showcase helper that builds a `Redb` node cache, see [Caching](/indexing/caching/). 5. `then(record_relative_path(..))` stores the file path, relative to the root, as metadata. 6. `then_chunk` splits each file along syntax boundaries with tree-sitter. `ChunkByLanguage` is the showcase’s own wrapper that picks a Swiftide `ChunkCode` per language. One file becomes several nodes. 7. `MetadataQACode` asks the LLM which questions each chunk answers and stores them as metadata. 8. `then_in_batch` with `Embed` embeds each chunk and its metadata with FastEmbed, on the central processing unit (CPU). `with_batch_size` sets 32 nodes per call. Without it, a batch holds 256 nodes. 9. `then_store_with` writes each node to storage. `codebase.vector_store()` builds a `LanceDB` store, see [Storage](/indexing/storage/). 10. `run().await` drives the stream to the end and returns the run’s statistics as `stats`. ## Nodes A node is a `Node`. It carries one piece of content and everything known about it. | Field | Holds | | --------------------------- | ------------------------------------------------------------------------------- | | `path` | The source path. | | `chunk` | The content, of type `T`. | | `metadata` | Key and value pairs, sorted by key. | | `vectors`, `sparse_vectors` | Embeddings, one per embedded field. `Embed` and `SparseEmbed` fill them. | | `embed_mode` | Whether the chunk and metadata are embedded together, each on its own, or both. | | `offset` | A running position that `ChunkCode` counts from 0 over the chunks it keeps. | | `parent_id` | The `id()` of the source node when the first chunker split it. | Since 0.31 the pipeline is generic. A `Pipeline` streams `Node`. A chunk is any type that implements the `Chunk` marker trait, which requires `Clone + Send + Sync + Debug + AsRef<[u8]> + 'static`. Most built-in steps take `TextNode`, which is `Node`. `id()` returns a universally unique identifier (UUID) computed from the path and the chunk, so it changes when either changes. The node cache keys on this id, see [Cache keys](/indexing/caching/#cache-keys). ## Steps | Step | Method | Trait | | ----------------- | ------------------------------------------------ | --------------------------------- | | loader | `Pipeline::from_loader`, `Pipeline::from_stream` | `Loader` | | filter | `filter` | a closure over `&Result>` | | node cache | `filter_cached` | `NodeCache` | | transformer | `then` | `Transformer`, or a closure | | chunker | `then_chunk`, `then_expand` | `ChunkerTransformer` | | batch transformer | `then_in_batch` | `BatchableTransformer` | | embedder | `then_in_batch(Embed::new(model))` | `EmbeddingModel` | | storage | `then_store_with` | `Persist` | A chunker returns a stream of nodes for each node. `then_expand` behaves like `then_chunk` with a different name in traces. Use it when you split a node for a reason other than chunking. A step can change the node type. `then` with a transformer whose `Output` is `U` turns a `Pipeline` into a `Pipeline`. Nodes keep flowing after storage. ## Execution The pipeline methods only describe the stream, but their arguments run while you build the chain. In the snippet, `codebase.node_cache()?` creates the Redb file and `embedding::local_model()?` loads the FastEmbed model, which downloads it on first use. `run` takes the pipeline by value and does the work: 1. It calls `setup` on every storage backend. LanceDB creates its table here if it is missing. 2. It pulls nodes through the stream until the stream ends. Some steps spawn Tokio tasks, so `run` needs a Tokio runtime. 3. It marks the source nodes in the node cache, but only after the stream ended without an error. 4. It logs `Pipeline completed` at info level and returns the same numbers as `PipelineStats`, see [Statistics](/indexing/streams-and-concurrency/#statistics). `run` returns `Result`, not the nodes. The showcase reads the number of stored chunks from `stats.nodes_stored`. It counts files with pass-through closures such as `.then(files_found.count())`, because the statistics count nodes at the end of the stream, after chunking. `then`, `then_chunk`, `then_in_batch` and storage work on several nodes at once and pass them on as they finish, see [Streams and concurrency](/indexing/streams-and-concurrency/). The showcase passes `.concurrency(concurrent_requests)` to `MetadataQACode` to bound requests to the LLM. ### Errors and the node cache The first error stops the run, and `run` returns it, see [Errors](/indexing/streams-and-concurrency/#errors). It is the error the step failed with, so a `LanguageModelError` can be recovered with `downcast_ref`. A source node is cached only when at least one of its chunks reached the end and none of them failed, and [When nodes are cached](/indexing/caching/#when-nodes-are-cached) covers the consequences. ## Output Two runs on the `swiftide-agents` crate. `claude-haiku-5-5` wrote the questions, through an OpenAI-compatible gateway to Amazon Web Services (AWS) Bedrock. FastEmbed `jina-embeddings-v2-base-code` computed the embeddings. ```text $ ./target/debug/ask-your-codebase index swiftide-agents --model claude-haiku-5-5 Indexing Rust files in swiftide-agents ... Indexed 34 chunks from 14 new or changed files (14 files found, 0 unchanged) in 91.9s ``` The second run, with no file changed: ```text $ ./target/debug/ask-your-codebase index swiftide-agents --model claude-haiku-5-5 Indexing Rust files in swiftide-agents ... Indexed 0 chunks from 0 new or changed files (14 files found, 14 unchanged) in 0.6s ``` The node cache drops all 14 files at step 4, so no prompt is sent. ## Feature flags `Pipeline`, `FileLoader` and `Embed` need no feature flag. The showcase turns off the defaults and turns on `tree-sitter`, `fastembed`, `lancedb`, `redb` and `openai`. To add the same features to your crate: ```sh cargo add swiftide@0.34 --no-default-features --features tree-sitter,fastembed,lancedb,redb,openai ``` TLS needs no setup, because swiftide’s HTTP clients bring their own rustls crypto provider ([TLS](/integrations/overview/#tls)). [Feature flags](/reference/feature-flags/) lists every flag. ## Reference * [`indexing::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`PipelineStats`](https://docs.rs/swiftide/0.34/swiftide/indexing/statistics/struct.PipelineStats.html) * [`Node`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Node.html) * [`TextNode`](https://docs.rs/swiftide/0.34/swiftide/indexing/type.TextNode.html) * [`EmbedMode`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbedMode.html) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`Loader`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Loader.html) * [`Transformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Transformer.html) * [`ChunkerTransformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChunkerTransformer.html) * [`BatchableTransformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.BatchableTransformer.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html) * [`NodeCache`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.NodeCache.html) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.SparseEmbed.html) * [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkCode.html) * [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQACode.html) * [`Redb`](https://docs.rs/swiftide/0.34/swiftide/integrations/redb/struct.Redb.html) * [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) ## Next [Loading data](/indexing/loading/) # Storage > Store indexed nodes with then_store_with in LanceDB, Qdrant, PgVector, DuckDB, Redis, Kafka or MemoryStorage, and write your own storage by implementing the Persist trait. `then_store_with` writes every node to a storage backend. You end up with nodes in LanceDB or Qdrant, a clear picture of what each backend keeps of a node, and a storage of your own built on the `Persist` trait. ## LanceDB The ask-your-codebase showcase keeps its vectors in an embedded LanceDB table, in a directory next to the code. There is no server to run. Enable the `lancedb` feature flag: ```sh cargo add swiftide@0.34 --features lancedb ``` examples/ask-your-codebase/src/codebase.rs ```rust /// Remembers which files were fully indexed, so unchanged files are skipped next time. pub fn node_cache(&self) -> Result { std::fs::create_dir_all(&self.data_dir)?; Redb::builder() .database_path(self.data_dir.join("cache.redb")) .table_name("indexed_files") .build() } /// Embedded vector store with one vector per chunk and the source path next to it. pub fn vector_store(&self) -> Result { let uri = self.lancedb_dir(); LanceDB::builder() .uri(uri.to_str().context("Index path is not valid UTF-8")?) .table_name("code_chunks") .vector_size(embedding::DIMENSIONS) .with_vector(EmbeddedField::Combined) .with_metadata(PATH_FIELD) .build() } ``` `vector_store` builds the storage; `node_cache` belongs to [Caching](/indexing/caching/). The showcase pipeline passes the storage to `then_store_with`, after the embedding step. * `uri` is the directory that holds the table. It is required unless you pass a `connection_pool`. * `table_name` defaults to `swiftide`. * `vector_size` must match the output of your embedder. The showcase uses 768, the size of its FastEmbed code model. * `with_vector(EmbeddedField::Combined)` adds a column for the embedding of the chunk together with its metadata. * `with_metadata(PATH_FIELD)` adds a column for the `path` metadata field. LanceDB stores only what you configure: the node id, the chunk, each configured vector and each metadata field named with `with_metadata`. The node’s own `path` is not a column. The showcase stores the path relative to the codebase root as metadata, so answers can cite their sources. Metadata columns hold text, so a value that is not a JSON string is stored as null. Before the first node arrives, setup creates the table if it does not exist. Nodes are written in batches of 256 with a merge on `id`, so storing the same node twice updates its row. See [LanceDB](/integrations/lancedb/) for retrieval and remote storage. ## Qdrant The docs-qa showcase stores a dense and a sparse vector for every chunk in a Qdrant collection. Enable the `qdrant` feature flag: ```sh cargo add swiftide@0.34 --features qdrant ``` examples/docs-qa/src/store.rs ```rust let qdrant = Qdrant::builder() .batch_size(BATCH_SIZE) .vector_size(DENSE_VECTOR_SIZE) // One dense and one sparse vector, both over the chunk *and* its metadata .with_vector(EmbeddedField::Combined) .with_sparse_vector(EmbeddedField::Combined) .collection_name(&collection.0) .build()?; ``` Qdrant keeps more of the node than LanceDB. Each point’s payload holds every metadata field, plus `path`, `content` (the chunk) and `last_updated_at`. The point id is the node id. Nodes without vectors fail with “Node without vectors”. Setup creates the collection when it is missing. An existing collection is kept as it is, even when its vector configuration differs from the builder. The client reads `QDRANT_URL` (default `http://localhost:6334`, gRPC) and `QDRANT_API_KEY`. The default batch size is 50; docs-qa raises it to 64. A debug build of docs-qa split four markdown pages into 43 chunks and stored them in Qdrant (metadata from `claude-haiku-5-5`, embeddings from local FastEmbed): ```text Indexed 43 chunks into `docs-qa` in 71.7s ``` [Qdrant](/integrations/qdrant/) covers named vectors, hybrid search and filters. ## Backends Every backend sits behind its own feature flag, except `MemoryStorage`: | Storage | Feature flag | What it stores | Default batch size | Retrieval | | --------------- | ------------ | -------------------------------------------------------------- | ------------------ | --------- | | `LanceDB` | `lancedb` | id, chunk, configured vectors, `with_metadata` fields | 256 | yes | | `Qdrant` | `qdrant` | vectors; payload with all metadata, path, chunk | 50 | yes | | `PgVector` | `pgvector` | id, chunk, configured vectors, `with_metadata` fields as JSONB | 50 | yes | | `Duckdb` | `duckdb` | `uuid`, chunk, path, configured vectors; no metadata | none, node by node | yes | | `Redis` | `redis` | the node as JSON under `:` | 10 | no | | `Kafka` | `kafka` | the node as JSON, keyed `:`, on a topic | 32 | no | | `MemoryStorage` | none | the whole node, in memory | none, node by node | no | PgVector fails to store a node that lacks a field named with `with_metadata`; see [PgVector](/integrations/pgvector/). Kafka’s setup fails when the topic does not exist, unless you set `create_topic_if_not_exists`. Kafka is also a loader; see [Kafka and Fluvio](/indexing/loading/#kafka-and-fluvio). DuckDB stores node by node and ignores its builder’s `batch_size`. It rejects a node that lacks a vector for a configured field. When a row with the same `uuid` exists, DuckDB keeps the old row unless you set `upsert_vectors`, which needs DuckDB 1.2.1 or later. Redis works as a key-value store for nodes. `persist_key_fn` changes the key and `persist_value_fn` the value. Both take a plain `fn` pointer, so a closure that captures variables does not compile. The example writes only the metadata and raises the batch size from 10 to 50: examples/index_into_redis.rs ```rust indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["rs"])) .then_chunk(ChunkCode::try_for_language_and_chunk_size( "rust", 10..2048, )?) .then_store_with( // By default the value is the full node serialized to JSON. // We can customize this by providing a custom function. Redis::try_build_from_url(&redis_url)? .persist_value_fn(|node| Ok(serde_json::to_string(&node.metadata)?)) .batch_size(50) .build()?, ) .run() .await?; ``` ### DuckDB No swiftide example uses DuckDB, so this section describes the builder without code. `Duckdb::builder()` returns a `DuckdbBuilder` with these methods: * `connection` takes anything that converts into a `duckdb::Connection`. Swiftide does not re-export the `duckdb` crate, so add it to your crate yourself. * `with_vector(field, size)` adds a vector column of `size` floats for an `EmbeddedField`, such as `EmbeddedField::Combined`. * `table_name` defaults to `swiftide`, and `cache_table` to `swiftide_cache`. `cache_key_prefix` prefixes the node cache keys, which also lets you invalidate the cache by hand. * `schema` replaces the SQL that loads the extensions and creates the table. The upsert statement is built from the table name and vectors, so keep the column names when you change it. `upsert_vectors` is the flag described above. `Duckdb` implements `Persist` for `then_store_with`, `NodeCache` for `filter_cached`, and `Retrieve` for three search strategies: `SimilaritySingleEmbedding`, `HybridSearch` and `CustomStrategy`. The setup installs and loads the `vss` and `fts` extensions before it creates the table. Retrieved documents carry `id` and `path` as metadata, since no other metadata is stored. See [`DuckdbBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/duckdb/struct.DuckdbBuilder.html) for every method. ## MemoryStorage `MemoryStorage` lives in `swiftide::indexing::persist` and needs no feature flag. It suits tests and experiments: ```rust let storage = MemoryStorage::default(); Pipeline::from_stream(vec![ TextNode::new("Swiftide is a Rust library for building LLM applications."), TextNode::new("An indexing pipeline loads, transforms and stores nodes."), ]) .then(|mut node: TextNode| -> anyhow::Result { let words = node.chunk.split_whitespace().count(); node.metadata.insert("Words", words); Ok(node) }) // Clones share the same map, so keep one to read from later .then_store_with(storage.clone()) .run() .await?; for node in storage.get_all_values().await { println!("{:?} words: {}", node.metadata.get("Words"), node.chunk); } ``` Its keys are an incrementing counter, not the node id, so storing the same node twice keeps two copies. `get(key)` reads one node, `get_all_values()` returns all nodes, and `get_all()` returns them with their keys. It has no retrieval for query pipelines. ## Batching A storage chooses how it receives nodes through `batch_size`. When it returns `Some(n)`, the pipeline groups up to `n` nodes and calls `batch_store`. When it returns `None`, the pipeline calls `store` for each node. In both cases, up to the pipeline’s concurrency of these calls run at once. Storage has no concurrency setting of its own, and [Concurrency](/indexing/streams-and-concurrency/#concurrency) explains when the pipeline value is read. For how this differs from the embedder’s batch size, see [Batch sizes](/indexing/embedding/#batch-sizes). Stored nodes flow on, so you can chain a second `then_store_with` to write to two backends. If `setup` fails, for example because Qdrant is unreachable, `run` returns that error. To keep going after a failed write, see [Errors](/indexing/streams-and-concurrency/#errors). ## Custom storage Implement `Persist` to write nodes anywhere else. This one appends each node as a line of JSON: ```rust /// Appends every node as one line of JSON to a file. #[derive(Debug, Clone)] pub struct JsonLines { path: PathBuf, // Batches are stored concurrently; one writer at a time keeps lines whole write_lock: Arc>, } impl JsonLines { pub fn new(path: impl Into) -> Self { Self { path: path.into(), write_lock: Arc::default(), } } async fn append(&self, nodes: &[TextNode]) -> anyhow::Result<()> { let mut lines = String::new(); for node in nodes { lines.push_str(&serde_json::to_string(node)?); lines.push('\n'); } let _guard = self.write_lock.lock().await; let mut file = OpenOptions::new() .create(true) .append(true) .open(&self.path) .await?; file.write_all(lines.as_bytes()).await?; Ok(()) } } #[async_trait] impl Persist for JsonLines { type Input = String; type Output = String; // Runs once, before the first node arrives async fn setup(&self) -> anyhow::Result<()> { if let Some(dir) = self.path.parent() { tokio::fs::create_dir_all(dir).await?; } Ok(()) } // Used when `batch_size` returns `None` async fn store(&self, node: TextNode) -> anyhow::Result { self.append(std::slice::from_ref(&node)).await?; Ok(node) } // Used when `batch_size` returns `Some`; pass the stored nodes on async fn batch_store(&self, nodes: Vec) -> IndexingStream { self.append(&nodes).await.map(|()| nodes).into() } fn batch_size(&self) -> Option { Some(100) } } ``` * `Input` is the chunk type the storage accepts, and `Output` the chunk type it passes on. Both are `String` for text nodes. * `batch_store` returns an `IndexingStream`. Return the stored nodes so later steps receive them, or an error. * `Persist` requires `Debug`, `Clone`, `Send` and `Sync`, so fields like `Rc` or `RefCell` do not compile. Implement it with `#[async_trait]`. Batches are stored concurrently, so guard anything they share. Use the storage like any other: ```rust Pipeline::from_stream(vec![TextNode::new("Stored as a line of JSON.")]) .then_store_with(JsonLines::new("index/nodes.jsonl")) .run() .await?; ``` ## Re-indexing A node’s id is computed from its path and its chunk, as [Cache keys](/indexing/caching/#cache-keys) explains. When you index an unchanged chunk again, LanceDB, Qdrant, PgVector and DuckDB find the same id and keep one row. DuckDB keeps the old row as it is unless `upsert_vectors` is set. When a chunk changes, it gets a new id and a new row, and the row of the old version stays. Swiftide does not delete rows for you. docs-qa’s `--recreate` flag drops the Qdrant collection before indexing. For incremental updates, delete a file’s rows yourself before you index its new version, and skip unchanged files with a [node cache](/indexing/caching/). ## Reference * [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html) * [`Pipeline::then_store_with`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.then_store_with) * [`IndexingStream`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.IndexingStream.html) * [`MemoryStorage`](https://docs.rs/swiftide/0.34/swiftide/indexing/persist/struct.MemoryStorage.html) * [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`PgVector`](https://docs.rs/swiftide/0.34/swiftide/integrations/pgvector/struct.PgVector.html) * [`Duckdb`](https://docs.rs/swiftide/0.34/swiftide/integrations/duckdb/struct.Duckdb.html) * [`Redis`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.Redis.html) * [`Kafka`](https://docs.rs/swiftide/0.34/swiftide/integrations/kafka/struct.Kafka.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [ask-your-codebase source](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase) * [docs-qa source](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa) ## Next [Streams and concurrency](/indexing/streams-and-concurrency/) # Streams and concurrency > Set how many nodes an indexing pipeline processes at once, throttle it, split and merge its stream, and drop failed nodes without losing good ones. You set how many nodes each step of an indexing pipeline handles at once, how fast they reach a slow step, which steps they pass through, and what happens to the ones that fail. The pipeline is a stream of nodes, and every step is a stage on that stream. The snippets use `OpenAI`, which needs the `openai` feature flag, and `ChunkCode`, which needs the `tree-sitter` feature flag: ```sh cargo add swiftide@0.34 --features openai,tree-sitter ``` ## Concurrency ```rust indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) // Applies to every step added after this call .with_concurrency(8) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) // At most one chunk every 50ms enters the next step .throttle(Duration::from_millis(50)) // Its own concurrency wins over the pipeline's 8 .then(MetadataQAText::new(openai.clone()).with_concurrency(2)) .then_in_batch(Embed::new(openai).with_batch_size(64)) .then_store_with(MemoryStorage::default()) .run() .await?; ``` Each transformer, chunker and storage step spawns a Tokio task per node, or per batch for batch transformers and batch storage, and keeps up to its concurrency of them running. Results leave a step as they finish, so the order of nodes is not preserved. `with_concurrency` sets the pipeline value. The default is the number of logical cores of the machine. A step reads the value when you add it, so call `with_concurrency` before the steps it should apply to. A transformer, batch transformer or chunker can bring its own limit through `concurrency()`. When that returns a value, it wins over the pipeline’s. The built-in large language model (LLM) transformers and `Embed` set it with `with_concurrency`, as `MetadataQAText` does above. Storage has no such override and always uses the pipeline value. `throttle` lets at most one node through per duration, at the point where you put it. Here it caps the chunks that reach the LLM at 20 per second, whatever the concurrency. ## Split and merge ```rust let (markdown, code) = indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["md", "rs"])) .split_by(|result| { result .as_ref() .is_ok_and(|node| node.path.extension().is_some_and(|ext| ext == "md")) }); let markdown = markdown.then_chunk(ChunkMarkdown::from_chunk_range(100..2048)); let code = code.then_chunk(ChunkCode::try_for_language("rust")?); // One set of statistics for both branches let stats = markdown .merge(code) .then_store_with(MemoryStorage::default()) .run() .await?; ``` `split_by` turns one pipeline into two. The predicate gets each item as a `Result`, errors included. Items for which it returns true go to the first pipeline, the rest to the second. Here, errors go to the code branch. `merge` joins two pipelines with the same node type into one stream. Node cache registrations and recorded failures of both sides are combined, so a `filter_cached` inside either branch still marks its sources and failures in the other branch keep them out of the cache. Each branch of `split_by` gets its own statistics collector, seeded with a copy of what was collected before the split, so a branch you run on its own reports only its own nodes. A collector handle taken before the split sees neither branch. `merge` keeps the collector of the pipeline you call it on, so `run` after the merge returns one `PipelineStats` for both branches. ### Split gotchas * `split_by` is not lazy. It spawns a task that starts reading the stream at once and sends each item into one of two channels that hold 1000 items each. * When one branch is not read, its channel fills up and the split stops, which also stalls the other branch. Run both branches together: merge them, or run them at the same time with `tokio::join!`. Running one to completion before starting the other can hang once the waiting branch has 1000 items. * If a branch is dropped, the split task panics on its next send to it. The panic stays inside that task, and the other branch then ends early without an error. * `merge` keeps the storage setup of the pipeline you call it on and drops that of the pipeline you pass in. Add storage after the merge, as above, or to the branch you call `merge` on. ## Errors ```rust indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .filter(|result| match result { Ok(node) => !node.chunk.trim().is_empty(), // Keep errors, so `log_errors` sees them Err(_) => true, }) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) .then(MetadataQAText::new(openai.clone())) // Drop failed nodes before the batch step, so an error does not take its batch with it .log_errors() .filter_errors() .then_in_batch(Embed::new(openai).with_batch_size(64)) .log_nodes() .then_store_with(MemoryStorage::default()) .run() .await?; ``` `run` stops at the first error a step returns and returns that error, and a node cache marks nothing from a failed run. The steps above drop failed nodes so the run continues. The returned error is the one the step failed with, even when a batch step follows it. A `LanguageModelError` from an LLM transformer can be recovered with `downcast_ref` or found in `chain()`, see [Errors, backoff and usage](/models/errors-backoff-and-usage/). * `filter` keeps an item when the closure returns true. It sees nodes and errors, so it can drop specific errors as well as nodes. This one drops empty files and keeps every error. * `log_errors` logs each error at ERROR level and passes it on. * `filter_errors` drops errors without logging them. * `log_nodes` logs each node at DEBUG level. `log_all` is `log_errors` followed by `log_nodes`. Order matters. `filter_errors` before `log_errors` leaves nothing to log. Each pair covers only the steps before it, and errors from steps added after it still abort the run. Add another pair after `then_store_with` if a failed write should not stop the run. ### Errors before a batch step A batch step collects nodes until it has a full batch. Those are `then_in_batch`, and `then_store_with` when the storage has a batch size. When an error reaches it, the step passes the error on and drops the nodes it has collected so far, without a trace. A `filter_errors` after the batch step keeps the run going, but those good nodes are already gone. With a batch size of 64, one failed LLM call can drop up to 63 good chunks. Those dropped nodes are not recorded as failed. When other chunks of the same source complete, the node cache marks that source and the next run skips it. Put a `filter_errors` pair before every batch step, as the snippet does before `then_in_batch`. The source of a node that fails in a transformer stays out of the node cache, so the next run retries it. [Caching](/indexing/caching/) covers when sources are marked. ## Statistics `run` returns the `PipelineStats` of the run, with node counts, token usage and timing, as `let stats = pipeline.run().await?;`. It also logs them as the INFO event `Pipeline completed`. `run` takes the pipeline by value, and `stats()` before `run` returns the empty initial statistics. To watch a run while it is going, clone the handle from `stats_collector()` before you call `run`. [Pipeline statistics](/observability/metrics-and-statistics/#pipeline-statistics) shows both. ## Streams Loaders, chunkers, batch transformers and batch storage return their nodes as an `IndexingStream`. `IndexingStream::iter` takes any iterator of results, `IndexingStream::from_nodes` a `Vec` of nodes, and `IndexingStream::empty` makes an empty stream. Use them to return nodes from your own chunkers and batch transformers. When you build a pipeline with `Pipeline::from_stream`, a `Result>` that is an error and an `anyhow::Error` each become a stream with that one error item. A Tokio channel’s capacity applies backpressure: `send` waits while the pipeline is behind. [Loading](/indexing/loading/#nodes-in-memory) shows `from_stream` with a vector and a channel. ## Reference * [`Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) (`with_concurrency`, `throttle`, `then`, `then_chunk`, `then_in_batch`, `then_store_with`, `split_by`, `merge`, `filter`, `filter_cached`, `filter_errors`, `log_all`, `log_errors`, `log_nodes`, `from_stream`, `run`, `stats`, `stats_collector`) * [`IndexingStream`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.IndexingStream.html) * [`Transformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Transformer.html) (`concurrency`) * [`BatchableTransformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.BatchableTransformer.html) (`concurrency`) * [`ChunkerTransformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChunkerTransformer.html) (`concurrency`) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`ChunkMarkdown`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkMarkdown.html) * [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkCode.html) * [`MetadataQAText`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQAText.html) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`MemoryStorage`](https://docs.rs/swiftide/0.34/swiftide/indexing/persist/struct.MemoryStorage.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.OpenAI.html) ## Next [Code-aware indexing](/indexing/code-aware-indexing/) # Transformers and metadata > Add metadata to nodes with closures, the built-in LLM metadata transformers and their prompts, and your own transformers and batch transformers. You add metadata to nodes with closures, with the built-in transformers that ask a large language model (LLM), or with a transformer you write yourself. Metadata travels with each node to the embedder, so a retriever can match on it and an answer can cite it. ## Closures The [ask-your-codebase](/showcases/ask-your-codebase/) showcase stores each file’s path, relative to the codebase root, on every node. examples/ask-your-codebase/src/index.rs ```rust /// Stores the file path, relative to the codebase root, as metadata on the node. /// /// The vector store keeps this field next to each chunk, so answers can cite their sources. fn record_relative_path( codebase: Codebase, ) -> impl Fn(TextNode) -> Result + Clone + Send + Sync + 'static { move |mut node| { let path = codebase .relative_path(&node.path) .to_string_lossy() .into_owned(); node.metadata.insert(PATH_FIELD, path); Ok(node) } } ``` A closure from `TextNode` to `Result` is a transformer if it is `Clone`, `Send`, `Sync` and `'static`, so it cannot borrow local data. You add it with `.then(...)`. The pipeline runs it on many nodes at once and does not keep their order. `node.metadata` is a `Metadata` map. `insert` takes a key that converts into a `String` and a value that converts into a `serde_json::Value`. LanceDB and PgVector store a field only after you declare it with `with_metadata`, which is why the showcase calls `.with_metadata(PATH_FIELD)`. ## Built-in metadata transformers Swiftide ships six transformers that prompt the LLM per node and store the reply as metadata: | Transformer | Metadata field | Feature flag | | --------------------- | ------------------------------ | ------------- | | `MetadataQAText` | `Questions and Answers (text)` | none | | `MetadataQACode` | `Questions and Answers (code)` | `tree-sitter` | | `MetadataSummary` | `Summary` | none | | `MetadataTitle` | `Title` | none | | `MetadataKeywords` | `Keywords` | none | | `CompressCodeOutline` | `Outline` | `tree-sitter` | Each module exports the field name as a `NAME` constant, such as `metadata_title::NAME`. `CompressCodeOutline` skips nodes without an `Outline` field; [Code-aware indexing](/indexing/code-aware-indexing/) shows how to fill it. The next example needs the `openai` and `qdrant` feature flags. examples/index_markdown_lots_of_metadata.rs ```rust indexing::Pipeline::from_loader(FileLoader::new("README.md").with_extensions(&["md"])) .with_concurrency(1) .then_chunk(ChunkMarkdown::from_chunk_range(20..2048)) .then(MetadataQAText::new(openai_client.clone())) .then(MetadataSummary::new(openai_client.clone())) .then(MetadataTitle::new(openai_client.clone())) .then(MetadataKeywords::new(openai_client.clone())) .then_in_batch(Embed::new(openai_client.clone())) .log_all() .filter_errors() .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-examples") .build()?, ) .run() .await?; ``` `new(client)` uses the default prompt. `from_client(client)` returns the builder, where you set `prompt_template`, `concurrency` and, for the question transformers, `num_questions` (default 5). `builder()` and `Default` start without a client. After the chunker, each chunk gets its own prompt and reply. Before it, one prompt per file is enough, and chunking copies the reply to every chunk of that file. `concurrency(n)` on the builder, or `with_concurrency(n)`, limits the requests that step sends at once. [Concurrency](/indexing/streams-and-concurrency/#concurrency) explains how this interacts with the pipeline value. By default the embedder sees every metadata field as a `key: value` line in front of the chunk. [Embed mode](/indexing/embedding/#embed-mode) explains how to embed fields separately. ## Prompts The default prompts show the expected answer inside a code fence, and models tend to copy that fence into the metadata. The [docs-qa](/showcases/docs-qa/) showcase replaces the `MetadataTitle` and `MetadataKeywords` prompts: examples/docs-qa/src/index.rs ```rust // Swiftide's default metadata prompts show the expected format inside a code fence, and models // tend to copy the fence into the metadata. These ask for plain text and use the page title. // A code-only chunk is titled by the API it shows, because the reranker scores the title too. const TITLE_PROMPT: &str = "Write a short, descriptive title (at most 10 words) for this \ excerpt from the \"{{ node.metadata.Page }}\" documentation. If the excerpt is mostly code, \ name the API it demonstrates (a macro, trait or builder method), not what the example does. \ Reply with the title only: no quotes, no markdown, no code fences. {{ node.chunk }}"; const KEYWORDS_PROMPT: &str = "List up to 8 keywords for this excerpt from the \ \"{{ node.metadata.Page }}\" documentation: names of types, crates, features, tools, commands \ and concepts that literally appear in it. Reply with one comma-separated line only: no \ numbering, no markdown, no code fences. {{ node.chunk }}"; ``` The pipeline passes them to the builders with `prompt_template`: examples/docs-qa/src/index.rs ```rust indexing::Pipeline::from_loader(loader) .then_chunk(ChunkMarkdown::from_chunk_range( MIN_CHUNK_CHARS..MAX_CHUNK_CHARS, )) // A short title and keywords per chunk; both end up in the embedded text and help the // sparse (keyword) side of hybrid search in particular. .then( MetadataTitle::from_client(llm.clone()) .prompt_template(TITLE_PROMPT) .build()?, ) .then( MetadataKeywords::from_client(llm.clone()) .prompt_template(KEYWORDS_PROMPT) .build()?, ) .then_in_batch(Embed::new(embedders.dense.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_in_batch(SparseEmbed::new(embedders.sparse.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_store_with(qdrant.clone()) .run() .await ``` `prompt_template` takes anything that converts into a `Prompt`, such as a `&'static str` holding a Tera template. Each transformer adds the node as `node`, so `{{ node.chunk }}` is the chunk and `{{ node.metadata.Page }}` is a field that the docs-qa loader set. The question transformers also add `{{ questions }}`, and `MetadataQACode` adds `{{ outline }}` when the node has an `Outline` field. In the captured docs-qa run, four pages became 43 chunks, and the title and keywords steps made 86 calls to `claude-haiku-5-5` through an OpenAI-compatible gateway to Amazon Web Services (AWS) Bedrock. The whole run, including local embedding and the writes to Qdrant, took: ```text Indexed 43 chunks into `docs-qa` in 71.7s ``` The ask-your-codebase prompt asks for questions a developer would ask: examples/ask-your-codebase/src/index.rs ````rust /// Asks for the questions a developer would ask that a chunk answers. /// /// The default `MetadataQACode` prompt asks generic questions ("What does this code do?"). /// Questions about behaviour, phrased the way a developer would ask them, match real questions /// much better. const QUESTIONS_PROMPT: &str = r" # Task List the questions a developer could ask about this codebase that the code below answers, each with a short answer. They are stored with the code to help a search find it. Focus on behaviour: when and in which order things happen, what starts or ends a process, which conditions and configuration change the outcome, what happens on errors, and which extension points (traits, hooks, builders, options) exist and when they apply. # Constraints - Write {{ questions }} questions and answers. - Phrase questions the way a developer would ask them, e.g. 'When does the agent stop?'. - Name the relevant types, functions and variants in the answers. - Only use what can be derived from the code. - Respond only in this format: Q1: A1: # Code ``` {{ node.chunk }} ``` "; ```` See [Prompts and templates](/models/prompts-and-templates/) for Tera syntax. ## Default LLM client You can set one client on the pipeline instead of on each transformer. This example needs the `openai` feature flag. ```rust Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) // Set the client before the steps that use it .with_default_llm_client(openai) .then_chunk(ChunkMarkdown::from_chunk_range(100..1500)) .then(MetadataTitle::default()) .then(MetadataSummary::default()) .then( MetadataQAText::builder() .num_questions(3_usize) .concurrency(4_usize) .build()?, ) .then_store_with(MemoryStorage::default()) .run() .await ``` `then` hands the pipeline defaults to the transformer when you add the step, so call `with_default_llm_client` first. A client on the transformer takes precedence. Without either, each node fails with `No client provided`. ## Custom transformers A transformer is a type that implements `Transformer`. ```rust /// Stores the number of words in a chunk as the `Words` metadata field. #[derive(Clone)] pub struct WordCount; #[async_trait] impl Transformer for WordCount { type Input = String; type Output = String; async fn transform_node(&self, mut node: TextNode) -> anyhow::Result { let words = node.chunk.split_whitespace().count(); node.metadata.insert("Words", words); Ok(node) } } // `then` hands pipeline defaults to every transformer; this one has no use for them. impl WithIndexingDefaults for WordCount {} ``` `transform_node` receives one node and returns it, changed, or an error. The type must be `Clone`, `Send`, `Sync` and `'static`. The trait is an `async_trait`, so add the `async-trait` crate. Override `concurrency` to give the step its own limit. `then` also requires `WithIndexingDefaults`; an empty `impl` is enough. ### Transformer macro The built-in LLM transformers use `#[indexing_transformer]`. This one asks who a chunk is for: ```rust /// The macro adds items next to the struct, so each annotated struct gets its own module. pub mod metadata_audience { use async_trait::async_trait; use swiftide::{indexing::TextNode, indexing_transformer, traits::Transformer}; /// Asks the LLM who a chunk is written for and stores the reply as `Audience`. #[indexing_transformer( metadata_field_name = "Audience", default_prompt_file = "prompts/metadata_audience.prompt.md" )] pub struct MetadataAudience { /// Product name passed to the prompt as `{{ product }}`. #[builder(default = "\"swiftide\".to_string()")] product: String, } #[async_trait] impl Transformer for MetadataAudience { type Input = String; type Output = String; async fn transform_node(&self, mut node: TextNode) -> anyhow::Result { // The macro renders nothing for you: add the node and your own values here. let prompt = self .prompt_template .clone() .with_node(&node) .with_context_value("product", self.product.as_str()); // `prompt` uses this transformer's client, or the pipeline default. let audience = self.prompt(prompt).await?; node.metadata.insert(NAME, audience); Ok(node) } // The macro stores `with_concurrency(n)` but does not apply it. fn concurrency(&self) -> Option { self.concurrency } } } ``` `metadata_field_name` becomes the `NAME` constant. `default_prompt_file` goes to `include_str!`, relative to the source file. The macro adds `client`, `prompt_template` and `concurrency` fields, and generates `builder()`, `new`, `from_client`, `with_concurrency`, `Default` and the `WithIndexingDefaults` impl. `derive(skip_debug, skip_clone, skip_default)` turns off generated derives. You still implement `Transformer`. Build the prompt with `with_node(&node)` before you pass it to the generated `self.prompt(...)`, which uses the transformer’s client or the pipeline default. Override `concurrency()` as the example does, or `with_concurrency` has no effect. The macro adds a `hidden` module, `NAME` and `default_prompt()` next to the struct, so give each annotated struct its own module. `new` and `Default` unwrap the builder, so give each of your own fields `#[builder(default = ...)]`. Your crate needs `swiftide-core`, `anyhow` and `derive_builder` as direct dependencies; [Crates](/reference/crates/) has the `cargo add` line. ## Batch transformers A batch transformer takes a `Vec` of nodes and returns a stream of any length. This one drops repeated chunks: ```rust /// Drops chunks that repeat an earlier chunk in the same batch, such as copied licence headers. #[derive(Clone)] pub struct DropRepeatedChunks { batch_size: usize, } #[async_trait] impl BatchableTransformer for DropRepeatedChunks { type Input = String; type Output = String; async fn batch_transform(&self, nodes: Vec) -> IndexingStream { let mut seen = HashSet::new(); let unique: Vec = nodes .into_iter() .filter(|node| seen.insert(node.chunk.trim().to_owned())) .collect(); unique.into() } fn batch_size(&self) -> Option { Some(self.batch_size) } } impl WithBatchIndexingDefaults for DropRepeatedChunks {} ``` You add it with `then_in_batch`: ```rust Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(100..1500)) .then_in_batch(DropRepeatedChunks { batch_size: 64 }) .then(WordCount) .then_store_with(MemoryStorage::default()) .run() .await ``` The pipeline groups nodes into batches of `batch_size()`, or 256 when it returns `None`. `into()` turns a `Vec` into an `IndexingStream`. `then_in_batch` requires `WithBatchIndexingDefaults`, again with an empty `impl`. A closure from `Vec` to `IndexingStream` is a batch transformer too, with the pipeline batch size. `Embed` and `SparseEmbed` are batch transformers. Batches are formed in arrival order, so the example only catches repeats that land in the same batch. ## Node types `Transformer` and `BatchableTransformer` declare `Input` and `Output` types, both bound by `Chunk`. `then` returns a pipeline of the transformer’s `Output`, so a different `Output` changes the node type for every step after it. Closures and most built-in transformers, chunkers and storage backends take `String` nodes, so add a transformer that changes the node type after every step that needs `String`. ## Reference * [`Pipeline::then`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.then) * [`Pipeline::then_in_batch`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.then_in_batch) * [`Pipeline::with_default_llm_client`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.with_default_llm_client) * [`Pipeline::with_concurrency`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.with_concurrency) * [`Transformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Transformer.html) * [`BatchableTransformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.BatchableTransformer.html) * [`WithIndexingDefaults`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.WithIndexingDefaults.html) * [`WithBatchIndexingDefaults`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.WithBatchIndexingDefaults.html) * [`Node`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Node.html) and [`TextNode`](https://docs.rs/swiftide/0.34/swiftide/indexing/type.TextNode.html) * [`Chunk`](https://docs.rs/swiftide/0.34/swiftide/indexing/trait.Chunk.html) * [`Metadata`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Metadata.html) * [`IndexingStream`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.IndexingStream.html) * [`MetadataQAText`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQAText.html) * [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQACode.html) * [`MetadataSummary`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataSummary.html) * [`MetadataTitle`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataTitle.html) * [`MetadataKeywords`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataKeywords.html) * [`CompressCodeOutline`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.CompressCodeOutline.html) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) and [`SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.SparseEmbed.html) * [`indexing_transformer`](https://docs.rs/swiftide/0.34/swiftide/attr.indexing_transformer.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) ## Next [Embedding](/indexing/embedding/) # Errors, backoff, usage and tokens > Tell transient from permanent model errors, retry with LanguageModelWithBackOff, record token usage with on_usage, and estimate tokens with CharEstimator or tiktoken. By the end of this page your OpenAI-compatible or `AwsBedrock` client retries rate limits on its own, reports the tokens each call used, and you can estimate the size of a prompt before you send it. ```rust match openai.prompt("Summarise the attached log.".into()).await { Ok(answer) => println!("{answer}"), Err(LanguageModelError::ContextLengthExceeded(e)) => { eprintln!("prompt too long, shorten it: {e}"); } Err(LanguageModelError::TransientError(e)) => { eprintln!("rate limit or outage, try again later: {e}"); } Err(error @ LanguageModelError::PermanentError(_)) => return Err(error.into()), } ``` A failed model call returns a `LanguageModelError`. Each of its three variants wraps the underlying error: * `ContextLengthExceeded`: the input is longer than the model accepts. Shorten it before you send it again. * `TransientError`: the call may succeed if you try again later. * `PermanentError`: the call fails the same way every time. The OpenAI-compatible clients sort provider errors into these variants for you: * The identifier `context_length_exceeded` becomes `ContextLengthExceeded`. * Network errors and Hypertext Transfer Protocol (HTTP) status 408, 429 and 5xx are transient. * OpenRouter’s rate limit, overload, unavailable, server and timeout error types are transient. * Quota and billing errors are permanent, even when they arrive with status 429. * An error inside a successful response is read and sorted the same way. * A response that fails to parse and holds no such error is transient. * Invalid arguments and broken streams are permanent. `AwsBedrock` treats throttling, timeouts, unavailable services and network errors as transient. `Anthropic` treats only network errors on `prompt` as transient. Every other `Anthropic` error is permanent, including status 429. To build a variant yourself, call `LanguageModelError::transient`, `permanent` or `context_length_exceeded`. An `anyhow::Error` converts into a `PermanentError`. To return a variant with `?` or `.into()`, bind the whole `LanguageModelError` (`error @ ...`). The `Box` inside a variant does not convert into `anyhow::Error`. ### Errors from a pipeline When a step of an indexing pipeline fails on a model call, `run` returns an `anyhow::Error` with the `LanguageModelError` still inside. So you can tell a rate limit from a broken request. ```rust let result = indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) .then(MetadataQAText::new(openai.clone())) .then_in_batch(Embed::new(openai)) .then_store_with(MemoryStorage::default()) .run() .await; if let Err(error) = result { let model_error = error .chain() .find_map(|cause| cause.downcast_ref::()); match model_error { Some(LanguageModelError::TransientError(_)) => { eprintln!("the provider is overloaded, run the pipeline again later: {error}"); } _ => return Err(error), } } ``` `downcast_ref::()` on the error works too. Searching `chain()` also finds the model error when a step added context around it. ## Backoff ```rust let openai = LanguageModelWithBackOff::new( openai, BackoffConfiguration { initial_interval_sec: 2, multiplier: 2.0, randomization_factor: 0.5, max_elapsed_time_sec: 120, }, ); let answer = openai.prompt("Name one Rust web framework.".into()).await?; ``` `LanguageModelWithBackOff` wraps a client and retries calls that fail with a `TransientError`. On `prompt` and `complete`, a `ContextLengthExceeded` or `PermanentError` returns at once. The decorated client implements `SimplePrompt`, `ChatCompletion` and `EmbeddingModel` when the inner client does. You can pass it where one of those traits is expected, including `Agent::builder().llm(&openai)`. It never retries an `Anthropic` rate limit, because `Anthropic` marks it permanent. `BackoffConfiguration` has four fields: * `initial_interval_sec`: the wait before the first retry, in seconds. The default is 1. * `multiplier`: the factor the wait grows by after each attempt. The default is 2.0. * `randomization_factor`: random jitter on each wait. The default is 0.5, plus or minus half the interval. * `max_elapsed_time_sec`: the time budget in seconds, including time spent inside calls. The decorator returns the last error as soon as the next wait would pass it, often before the full budget. The default is 60. `Default::default()` gives you these defaults. Every failed attempt that the decorator retries logs a WARN event, `Language model request failed, retrying`, with the attempt number, the wait and the error with its causes. When the decorator gives up, it logs the final error once at ERROR, also for a permanent error that it never retried. Only `prompt` and `complete` log these events. For `complete_stream` the decorator does not send the request again. Your stream still receives every error. After each one, transient or permanent, the decorator waits for the next interval before it reads on. Once `max_elapsed_time_sec` is used up, the stream ends after the last error. The budget restarts after every successful item. OpenAI-compatible streams end after their first error, so the wait only delays that end. Two kinds of call get no retry. `EmbeddingModel` calls the inner client once. `StructuredPrompt` and methods of the client itself, such as `estimate_tokens`, do not exist on the decorator, so call them on the inner client. ## Usage callbacks ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .on_usage(|usage| { println!( "{} tokens in, {} out", usage.prompt_tokens, usage.completion_tokens ); Ok(()) }) .build()?; ``` A usage callback receives the `Usage` of each call: `prompt_tokens`, `completion_tokens` and `total_tokens`. Set it with `on_usage` on the builder of an OpenAI-compatible client (`openai` feature flag), `Anthropic` (`anthropic`) or `AwsBedrock` (`aws-bedrock`). On OpenAI-compatible clients, embedding calls report usage too. Use `on_usage_async` when the callback needs to await: ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .on_usage_async(|usage| { Box::pin(async move { // Write to a database or a metrics service here. println!("{} tokens", usage.total_tokens); Ok(()) }) }) .build()?; ``` The closure gets a borrowed `Usage` and returns a boxed future that may borrow it. Copy the numbers out first only if you hand them to a task that must be `'static`, such as one you spawn. The providers run the callback differently: * OpenAI-compatible clients spawn it. An `Err` is logged and the call still succeeds. * `AwsBedrock`, and `Anthropic` on calls that do not stream, await it. An `Err` from the callback fails the call. * `Anthropic` streaming calls spawn it and only log an `Err`. With the `metrics` feature flag, clients also emit usage counters. See [Metrics and statistics](/observability/metrics-and-statistics/). ## Normalized usage ```rust let request = ChatCompletionRequest::from(vec![ChatMessage::new_user("Hello")]); let response = openai.complete(&request).await?; if let Some(usage) = response.usage.as_ref().map(Usage::normalized) { let cached = usage .details .as_ref() .and_then(|details| details.input.cached_tokens) .unwrap_or(0); println!("{} tokens, {cached} from cache", usage.total_tokens); } ``` `Usage` has an optional `details` field with the provider’s own breakdown. `Usage::normalized` returns a `NormalizedUsage` with one shape for every provider: cached and audio tokens on the input side, and reasoning, audio and prediction tokens on the output side. Its `details` is `None` when the provider reported none of them. OpenAI fills the details from Responses application programming interface (API) calls and from Chat Completions calls that do not stream. `AwsBedrock` reports cached input tokens. Streamed Chat Completions, embeddings and `Anthropic` report the totals only. ## Token estimation ```rust let history = vec![ ChatMessage::new_system("You are a helpful assistant."), ChatMessage::new_user("What is a borrow checker?"), ]; let rough = CharEstimator.estimate(history.as_slice()).await?; ``` `CharEstimator` needs no feature flag. It divides the number of characters by 4, then adds 4 tokens per message and 3 per list of messages. `estimate` comes from the `EstimateTokens` trait. It accepts any `Estimatable`: a `&str`, a `String`, a `&Prompt` (which it renders first), a `&ChatMessage` or a `&[ChatMessage]`. For a `Vec`, pass a slice. For estimates closer to what OpenAI models see, enable the `tiktoken` feature flag: ```sh cargo add swiftide@0.34 --features openai,tiktoken ``` ```rust let tokenizer = TikToken::try_from_model("gpt-4o")?; let prompt = Prompt::from("Explain lifetimes in one paragraph."); let from_tokenizer = tokenizer.estimate(&prompt).await?; // The client's tokenizer is gpt-4o unless you set `tiktoken` on its builder. let from_client = openai.estimate_tokens(&prompt).await?; ``` `TikToken` encodes the text with the Byte Pair Encoding (BPE) tokenizer of the model you name in `try_from_model`. `TikToken::default()` uses the `gpt-4o` tokenizer. With the feature on, an OpenAI client also has `estimate_tokens`, which uses the client’s `TikToken`. That is `gpt-4o` whatever the prompt model, unless you set `tiktoken` on the builder. Both return estimates, not exact counts. They skip what they cannot read as text: image, document, audio and video parts of a message, and the text of reasoning messages. Each message still adds its 4 tokens, so an image-only or reasoning message counts as 4. ## Reference * [`LanguageModelError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.LanguageModelError.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`StructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.StructuredPrompt.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) (`llm`) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`BackoffConfiguration`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.BackoffConfiguration.html) * [`GenericOpenAIBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html) (`on_usage`, `on_usage_async`) * [`AnthropicBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.AnthropicBuilder.html) (`on_usage`, `on_usage_async`) * [`AwsBedrockBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrockBuilder.html) (`on_usage`, `on_usage_async`) * [`Usage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.Usage.html) * [`NormalizedUsage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.NormalizedUsage.html) * [`CharEstimator`](https://docs.rs/swiftide/0.34/swiftide/struct.CharEstimator.html) * [`EstimateTokens`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EstimateTokens.html) * [`Estimatable`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Estimatable.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`TikToken`](https://docs.rs/swiftide/0.34/swiftide/integrations/tiktoken/struct.TikToken.html) ## Next [Tracing](/observability/tracing/) # Images and multimodal input > Send images, and on some providers documents, audio and video, to a chat model with ChatMessage::new_user_with_parts and content parts. By the end of this page you send an image and a question to a chat model in one request and print its answer. The example uses the OpenAI client, which sits behind the `openai` feature flag: ```sh cargo add swiftide@0.34 --features openai ``` ```rust let bytes = std::fs::read("diagram.png")?; let message = ChatMessage::new_user_with_parts(vec![ ChatMessageContentPart::text("Describe this diagram in one sentence."), ChatMessageContentPart::image(ChatMessageContentSource::bytes(bytes, Some("image/png"))), ]); let request = ChatCompletionRequest::builder() .messages(vec![message]) .build()?; let response = openai.complete(&request).await?; println!("{}", response.message().unwrap_or_default()); ``` ## Content parts `openai` is an `OpenAI` client built as in [Models](/models/overview/). The prompt model you set with `default_prompt_model` must accept images, for example `gpt-4o-mini`. A client without a model fails with `Model not set`. `ChatMessage::new_user_with_parts` builds a `ChatMessage::UserWithParts` message. It holds a list of content parts (`ChatMessageContentPart`) instead of one string. Here the list has a text part and an image part. The client sends them in that order. Swiftide 0.33 added `UserWithParts`. It is a breaking change: if you match on `ChatMessage`, you need an arm for the new variant. Each media part points at its data through a source (`ChatMessageContentSource`). `ChatMessageContentSource::bytes` takes the raw file and an optional media type. The OpenAI client encodes the bytes as base64 and sends them as a data Uniform Resource Locator (URL), `data:image/png;base64,...`. You do not need a base64 crate yourself. `complete` is the method of the `ChatCompletion` trait, so import `swiftide::traits::ChatCompletion`. `SimplePrompt` only takes a text prompt, so it cannot carry images. ## Image sources A source can be a URL, a data URL, bytes, an Amazon Web Services (AWS) Simple Storage Service (S3) object or a provider file id: ```rust let by_url = ChatMessageContentPart::image("https://example.com/chart.png"); let by_data_url = ChatMessageContentPart::image("data:image/png;base64,iVBORw0KGgo="); let by_file_id = ChatMessageContentPart::image(ChatMessageContentSource::file_id("file-abc123")); let in_s3 = ChatMessageContentPart::image(ChatMessageContentSource::s3( "s3://my-bucket/chart.png", None::, )); let raw_bytes = ChatMessageContentPart::image_with_format(vec![0x89, 0x50, 0x4e, 0x47], "png"); ``` * A `&str` or `String` becomes a URL source. That covers both `https://` URLs and data URLs. * `ChatMessageContentSource::file_id` refers to a file you uploaded to the provider earlier. * `ChatMessageContentSource::s3` points at an object in S3. The second argument is the bucket owner. Without an owner, write `None::`, because a bare `None` leaves its type unknown and does not compile. * A `Vec` or `&[u8]` becomes a bytes source without a media type. * `ChatMessageContentPart::image_with_format` sets the format, such as `png`, explicitly. AWS Bedrock reads it first and otherwise infers the format from the media type or the file extension. The OpenAI clients ignore it. ## Other content parts Besides text and images, a message can carry document, audio and video parts. The helpers are `ChatMessageContentPart::document`, `document_with_name`, `audio` and `video`. A document with a file name looks like this: ```rust let message = ChatMessage::new_user_with_parts(vec![ ChatMessageContentPart::text("List the action items in this report."), ChatMessageContentPart::document_with_name( ChatMessageContentSource::bytes(pdf, Some("application/pdf")), "report.pdf", ), ]); ``` AWS Bedrock sends the name as the document name. The OpenAI Responses application programming interface (API) sends it as the file name of a bytes document. Few providers accept these parts, so check the table below before you use them. The `audio`, `document` and `video` helpers leave the format unset, and only images have a `_with_format` helper. To set a format on another part, build the variant directly: ```rust let audio = ChatMessageContentPart::Audio { source: ChatMessageContentSource::bytes(recording, Some("audio/wav")), format: Some("wav".into()), }; ``` ## Provider support Every client converts content parts to its provider’s format before it sends the request. A part or source the client cannot convert fails `complete` with `LanguageModelError::PermanentError`, before anything reaches the provider. `complete_stream` returns a stream instead, and the error is its first item. | Client | Image | Document | Audio | Video | | ------------------------------------------------- | --------------------------------------- | --------------------- | --------------------- | --------------------- | | OpenAI, Chat Completions (default) | URL, data URL, bytes | no | bytes, `mp3` or `wav` | no | | OpenAI, Responses API (`use_responses_api(true)`) | URL, data URL, bytes, file id | URL, bytes, file id | no | no | | AWS Bedrock (`aws-bedrock`) | bytes, base64 data URL, S3, `s3://` URL | same sources as image | same sources as image | same sources as image | | Anthropic | no | no | no | no | * Ollama, Groq, Gemini, OpenRouter and Dashscope are [OpenAI-compatible clients](/models/overview/#providers) and share the OpenAI conversion. Whether the endpoint accepts the image depends on the provider and the model you pick. * On OpenAI Chat Completions, audio must be a bytes source in `mp3` or `wav`. Without a format it is sent as `mp3`, and any other format fails. The media type does not change this. * AWS Bedrock accepts only base64 data URLs and `s3://` URLs. It rejects every other URL, such as `https://`, `http://` or `file://`, and it rejects file ids. * AWS Bedrock needs a format for images, audio and video. Pass a media type, use a file extension in the URL, or set the format explicitly. Otherwise the call fails with `Bedrock image format is required`. * An AWS Bedrock document without a resolvable format does not fail. It is sent as `txt`, so a PDF in bytes without a media type goes out as a text document. * An AWS Bedrock document part needs a non-empty text part in the same message. Bedrock drops empty text parts, so `text("")` does not count. * The Anthropic client (`anthropic` feature flag) accepts `UserWithParts` messages with text parts only. Any other part returns an error. ## Gotchas * Token estimates ignore every part except text. See [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/). * `ChatMessage::as_ref` returns an empty string for a `UserWithParts` message, unless the message has exactly one text part and nothing else. Code that logs or searches message text through `as_ref` skips these messages. * The `Debug` output of a source cuts data URLs to their first 32 characters of data, and bytes sources show only their length. Debug logs of the message history stay short. * For image parts without a media type, the OpenAI clients label bytes as `application/octet-stream`. Pass the real media type to `ChatMessageContentSource::bytes`. Documents on the Responses API and audio on Chat Completions do not send the media type, so there it has no effect. ## Reference * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`ChatMessageContentPart`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessageContentPart.html) * [`ChatMessageContentSource`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessageContentSource.html) * [`ChatCompletionRequest`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionRequest.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`LanguageModelError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.LanguageModelError.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) ## Next [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/) # Models > The model traits Swiftide uses, which providers implement each one, and how to build a client for OpenAI, Anthropic, Ollama or any OpenAI-compatible endpoint. By the end of this page you have a client for your model provider, and you know which trait each part of Swiftide expects from it. ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .default_embed_model("text-embedding-3-small") .build()?; let anthropic = Anthropic::builder() .default_prompt_model("claude-sonnet-4-5") .build()?; let ollama = Ollama::builder() .default_prompt_model("llama3.2") .default_embed_model("nomic-embed-text") .build()?; ``` Every provider has a client, and every client has a builder. `default_prompt_model` sets the model for prompts and chat completions. `default_embed_model` sets the model for embeddings. `OpenAI` reads its key from `OPENAI_API_KEY`. `Anthropic` reads `ANTHROPIC_API_KEY`. Ollama runs on your machine and needs no key. OpenAI and the other OpenAI-compatible clients have no default model. A call without one fails with a permanent error that says `Model not set`. `Anthropic` falls back to `claude-3-5-sonnet-20241022`, a retired model, so always set `default_prompt_model`. Each provider sits behind its own feature flag. Turn on the ones you use: ```sh cargo add swiftide@0.34 --features openai,anthropic,ollama ``` ## Model traits Swiftide never calls a provider by name. Each part of it asks for a trait, and you pass any client that implements that trait. The traits live in `swiftide::traits`. Import a trait to call its method on a client. | Trait | Method | Used by | | ---------------------- | --------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `SimplePrompt` | `prompt`: one prompt in, text out | Large Language Model (LLM) transformers such as `MetadataQAText`, the `GenerateSubquestions` query transformer, the `answers::Simple` answer generator, and `Pipeline::with_default_llm_client` | | `ChatCompletion` | `complete`: messages and tools in, a response out | agents, through `Agent::builder().llm(..)`, and task nodes in `swiftide::tasks` | | `EmbeddingModel` | `embed`: a list of texts in, dense vectors out | the `Embed` batch transformer and the `Embed` query transformer | | `SparseEmbeddingModel` | `sparse_embed`: a list of texts in, sparse vectors out | the `SparseEmbed` batch transformer and the `SparseEmbed` query transformer | | `StructuredPrompt` | `structured_prompt::`: one prompt in, a Rust value out | your own code, for typed responses | Failed calls return a `LanguageModelError`. `complete_stream` is the exception: it returns a stream, and errors arrive as items of that stream. See [Streaming](/models/overview/#streaming). ## Providers | Client (feature flag) | `SimplePrompt` | `ChatCompletion` | `EmbeddingModel` | `SparseEmbeddingModel` | `StructuredPrompt` | | ------------------------------------------------------------------------------------------------------------------ | -------------- | ---------------- | ---------------- | ---------------------- | ------------------ | | `OpenAI` (`openai`) | yes | yes | yes | no | yes | | `Ollama` (`ollama`), `Groq` (`groq`), `Gemini` (`gemini`), `OpenRouter` (`open-router`), `Dashscope` (`dashscope`) | yes | yes | yes | no | yes | | `Anthropic` (`anthropic`) | yes | yes | no | no | no | | `AwsBedrock` (`aws-bedrock`) | yes | yes | no | no | yes | | `FastEmbed` (`fastembed`) | no | no | yes | yes | no | `OpenAI` is `GenericOpenAI`. `Ollama`, `Groq`, `Gemini`, `OpenRouter` and `Dashscope` are `GenericOpenAI` with their own configuration type. So the aliases implement every trait `OpenAI` does, and you can pass them anywhere a model trait is expected. They are still different types. A field typed `OpenAI` does not accept an `Ollama`. The endpoint and the model decide what works at runtime. Not every provider serves embeddings or structured output. `AwsBedrock` talks to Amazon Web Services (AWS) Bedrock through the Converse application programming interface (API), in the `aws_bedrock_v2` module. `FastEmbed` runs embedding models locally. Setup for both is on [AWS Bedrock](/integrations/aws-bedrock/) and [FastEmbed](/integrations/fastembed/). ## Clients as trait objects ```rust let prompt: Arc = Arc::new(openai.clone()); let embed: Arc = Arc::new(openai.clone()); let answer = prompt.prompt("Name one Rust web framework.".into()).await?; let vectors = embed.embed(vec![answer]).await?; let agent = Agent::builder().llm(&openai).build()?; ``` Store a client as `Arc` or `Arc` when you pick the provider at runtime. Cloning a client is cheap. Every built-in client keeps its connection or model behind an `Arc`, so clones share it. `Agent::builder().llm(&openai)` takes a reference. It clones the client into a `Box`, so you can keep using `openai` afterwards. `Agent` needs the `swiftide-agents` feature flag on top of the provider flags. The `structured_prompt::` method is generic, so `StructuredPrompt` cannot be a trait object. [Structured output](/models/structured-output/) covers `DynStructuredPrompt`, the variant that can. To retry rate limits and server errors, wrap the client in `LanguageModelWithBackOff`. See [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/). ## OpenAI-compatible endpoints A server that speaks the OpenAI API works with the `OpenAI` client. Point the client at it with `api_base` and `api_key` on the builder. ```rust let local = OpenAI::builder() .api_base("http://localhost:8000/v1") .api_key("not-needed-locally") .default_prompt_model("qwen2.5-coder") .build()?; ``` With a single endpoint you can skip the code. The default `OpenAI` client reads `OPENAI_BASE_URL` and `OPENAI_API_KEY` from the environment when you build it. For a custom HTTP (Hypertext Transfer Protocol) client or another configuration type, pass an `async_openai::Client` to `client(..)` instead. Azure OpenAI needs that route with `GenericOpenAI`; [Azure OpenAI](/integrations/openai/#azure-openai) shows it. The hosted-provider clients have the base URL already set and read their key from the environment: | Client | Key variable | | ------------ | ---------------------------------------- | | `Ollama` | none | | `Groq` | `GROQ_API_KEY` | | `Gemini` | `GEMINI_API_KEY` | | `OpenRouter` | `OPENROUTER_API_KEY` | | `Dashscope` | `QWEN_API_KEY`, then `DASHSCOPE_API_KEY` | `OpenRouter` without a key does not return an error. It panics on the first request. More on these clients is on [OpenAI-compatible providers](/integrations/openai-compatible/). ## Request settings Each OpenAI-compatible client holds one `Options` value. Every request it sends carries those settings: ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .default_embed_model("text-embedding-3-small") .default_options( Options::builder() .max_completion_tokens(2_048) .reasoning_effort(ReasoningEffort::Low) .seed(42) .metadata(serde_json::json!({ "team": "docs" })) .dimensions(512), ) .parallel_tool_calls(Some(false)) .for_end_user("user-1234") .build()?; ``` `default_options` merges into the options the builder already has: a field you set replaces the earlier value. `for_end_user` and `parallel_tool_calls` are builder shortcuts for the `user` and `parallel_tool_calls` fields. Apart from `reasoning_features`, every field starts unset, and Swiftide only sends the fields you set. That includes `parallel_tool_calls`: set it to `Some(false)` to stop the model from asking for several tools in one turn. Some reasoning models reject the field, so leave it at `None` for them. | Field | Sent with | | -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ | | `temperature`, `max_completion_tokens`, `seed`, `presence_penalty`, `metadata`, `reasoning_effort` | `complete`, `prompt` and `structured_prompt` | | `user` | `complete`, `prompt`, `structured_prompt` and `embed`; not sent on the Responses API | | `dimensions` | `embed` only; text-embedding-3 and later models | | `reasoning_features` | the Responses API only, and only when `reasoning_effort` is set | `ReasoningEffort` is re-exported from `async-openai` at `swiftide::integrations::openai`. The Responses API (`use_responses_api(true)`) ignores `seed` and `presence_penalty` and logs a warning. It accepts `metadata` only as a flat map of string values. `reasoning_features` defaults to `true` on `OpenAI`. It asks for reasoning summaries and encrypted reasoning content. Turn it off with `reasoning_features(false)`. To change a built client, clone it and edit the copy through `options_mut`. Each clone has its own `Options`: ```rust let mut creative = openai.clone(); creative.options_mut().temperature = Some(1.2); creative.options_mut().presence_penalty = Some(0.5); ``` ### Extra request fields Some providers accept request fields that the OpenAI API does not have. `Options` has `extra_body`, a JavaScript Object Notation (JSON) map that Swiftide merges into the top level of each request: ```rust let mut extra = serde_json::Map::new(); extra.insert("top_k".into(), serde_json::json!(20)); let ollama = Ollama::builder() .default_prompt_model("llama3.2") .default_options(Options::builder().extra_body(extra)) .build()?; ``` Only `ChatCompletion` requests carry `extra_body`. `prompt`, `structured_prompt` and `embed` calls do not. So LLM transformers such as `MetadataQAText` and the `answers::Simple` answer generator send the request without the extra fields. ### AWS Bedrock settings `AwsBedrock` has its own `Options` in `swiftide::integrations::aws_bedrock_v2`. Set them with `AwsBedrockBuilder::default_options`, which merges the same way, or change them later with `options_mut`. | Field | Effect | | ------------------------------------------------------ | -------------------------------------------------------------------------- | | `max_tokens`, `temperature`, `top_p`, `stop_sequences` | the Converse inference configuration | | `tool_strict` | strict schema validation for tool calls; `true` when unset | | `anthropic_beta` | Anthropic beta headers, sent in the model request fields | | `reasoning_effort` | `output_config.effort` for Claude models: `Low`, `Medium`, `High` or `Max` | | `additional_model_request_fields` | any other model request fields, as an `aws_smithy_types::Document` | `reasoning_effort` here is the Bedrock module’s own `ReasoningEffort` enum, not the OpenAI one. For a model ID that contains `claude-opus-4-5`, Swiftide adds the `effort-2025-11-24` beta header itself. Behind an inference profile or Amazon Resource Name (ARN), add it to `anthropic_beta` yourself. With `anthropic_beta` or `reasoning_effort` set, `additional_model_request_fields` must be a JSON object, or the request fails with a permanent error. ## Responses API and reasoning `OpenAI` uses the Chat Completions API by default. `use_responses_api(true)` switches `complete`, `complete_stream`, `prompt` and `structured_prompt` to the OpenAI Responses API. For `OpenAI`, only that API returns the reasoning of a reasoning model: ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .use_responses_api(true) .default_options(Options::builder().reasoning_effort(ReasoningEffort::Medium)) .build()?; let mut agent = Agent::builder().llm(&openai).build()?; agent .query("Which crate in this workspace owns the agent loop?") .await?; for message in agent.history().await? { if let ChatMessage::Reasoning(item) = message { println!("reasoning summary: {:?}", item.summary); } } ``` With `reasoning_effort` set, reasoning arrives in the `reasoning` field of `ChatCompletionResponse` as `ReasoningItem` values. An agent adds each one to its message history as a `ChatMessage::Reasoning`, even when the response holds nothing else. On the next request the client sends the items back, so the model continues from its earlier reasoning. These requests set `store` to false, so the encrypted reasoning lives in your message history, not at OpenAI. The client sends back only items with `encrypted_content`. With `reasoning_features(false)` there is none, and nothing goes back. OpenAI returns summaries only to organizations verified for reasoning access. The Responses API does not send `extra_body`. `AwsBedrock` returns signed Claude reasoning as `ReasoningItem` values too, and replays them on the next turn. The `Anthropic` client leaves `ChatMessage::Reasoning` out of its requests. ## Streaming `complete_stream` sends a chat completion request and returns a `ChatCompletionStream` instead of one response. Each item is a `Result`. You read it with `StreamExt::next` from the `futures-util` crate. Swiftide does not re-export it: ```sh cargo add futures-util@0.3 ``` ```rust let request = ChatCompletionRequest::builder() .messages(vec![ChatMessage::new_user( "Explain borrowing in Rust in three sentences.", )]) .build()?; let mut stream: ChatCompletionStream = openai.complete_stream(&request).await; let mut printed = 0; let mut last = None; while let Some(response) = stream.next().await { let response = response?; let text = response.message().unwrap_or_default(); print!("{}", &text[printed..]); printed = text.len(); last = Some(response); } if let Some(usage) = last.and_then(|response| response.usage) { println!("\n{} tokens", usage.total_tokens); } ``` Every item is a whole `ChatCompletionResponse`, not a fragment. `message` holds all the text so far. `tool_calls` holds the tool calls rebuilt from the fragments so far. `delta` holds the latest change: `delta.message_chunk` is the new text, and `delta.tool_calls_chunk` maps each tool call index to its fragments so far. The snippet prints the part of `message` it has not printed yet, which works the same for every client. The last item is the complete response. Token usage comes in at the end of the stream, so read `usage` from the last item. A request that fails before the stream starts gives a stream with a single `Err` item. A client without its own streaming keeps the default `complete_stream`. It calls `complete` and returns a stream with that one result. An agent that streams calls `complete_stream` for you. You read its chunks in the `on_stream` hook. See [Streaming](/agents/streaming/). ### Deltas only By default every item clones the full response, which grows with the answer. The OpenAI-compatible clients can send only the delta. Turn off `stream_full` on the builder: ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .stream_full(false) .build()?; let mut stream = openai.complete_stream(&request).await; let mut complete = None; while let Some(response) = stream.next().await { let response = response?; if response.message.is_some() { complete = Some(response); continue; } if let Some(chunk) = response .delta .as_ref() .and_then(|d| d.message_chunk.as_deref()) { print!("{chunk}"); } } ``` With `stream_full(false)`, every item but the last carries only `id` and `delta`. `message`, `tool_calls` and `usage` are `None`. The last item is still the complete response, with `message`, `tool_calls` and `usage` set. The snippet uses `message` to tell the last item apart and prints `delta.message_chunk` for the others. `stream_full` defaults to `true`. It is a setting of `GenericOpenAI`, so it exists on `OpenAI`, `Ollama`, `Groq`, `Gemini`, `OpenRouter` and `Dashscope`, on the Chat Completions API and the Responses API alike. `Anthropic` and `AwsBedrock` have no such setting and always send the accumulated response. ## Reference * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`SparseEmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SparseEmbeddingModel.html) * [`StructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.StructuredPrompt.html) * [`DynStructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.DynStructuredPrompt.html) * [`LanguageModelError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.LanguageModelError.html) * [`GenericOpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`OpenAIConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.OpenAIConfig.html) * [`AzureConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.AzureConfig.html) * [`Options`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.Options.html) * [`OptionsBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.OptionsBuilder.html) * [`GenericOpenAIBuilder::for_end_user`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.for_end_user) * [`GenericOpenAIBuilder::parallel_tool_calls`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.parallel_tool_calls) * [`GenericOpenAI::options_mut`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html#method.options_mut) * [`ReasoningEffort` (OpenAI)](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/enum.ReasoningEffort.html) * [`GenericOpenAIBuilder::use_responses_api`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.use_responses_api) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`ReasoningItem`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ReasoningItem.html) * [`Agent::history`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html#method.history) * [`Anthropic`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.Anthropic.html) * [`Ollama`](https://docs.rs/swiftide/0.34/swiftide/integrations/ollama/type.Ollama.html) * [`Groq`](https://docs.rs/swiftide/0.34/swiftide/integrations/groq/type.Groq.html) * [`Gemini`](https://docs.rs/swiftide/0.34/swiftide/integrations/gemini/type.Gemini.html) * [`OpenRouter`](https://docs.rs/swiftide/0.34/swiftide/integrations/open_router/type.OpenRouter.html) * [`Dashscope`](https://docs.rs/swiftide/0.34/swiftide/integrations/dashscope/type.Dashscope.html) * [`AwsBedrock`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrock.html) * [`Options` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.Options.html) * [`ReasoningEffort` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/enum.ReasoningEffort.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder::llm`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.llm) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`ChatCompletion::complete_stream`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html#method.complete_stream) * [`ChatCompletionStream`](https://docs.rs/swiftide/0.34/swiftide/traits/type.ChatCompletionStream.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatCompletionResponseDelta`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponseDelta.html) * [`GenericOpenAIBuilder::stream_full`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.stream_full) ## Next [Prompts and templates](/models/prompts-and-templates/) # Prompts and templates > Build prompts with Prompt::from and Tera templates, register named templates, and replace the default prompts of transformers, answer generators and agents. By the end of this page you can fill a template with values, reuse named templates, and replace the built-in prompts that Swiftide sends to a Large Language Model (LLM). `Prompt` is always available. The snippets also use the OpenAI client, an agent and the tree-sitter transformers, and each sits behind a feature flag that is off by default: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents,tree-sitter ``` [Feature flags](/reference/feature-flags/) lists the rest. ```rust let prompt = Prompt::from("Explain {{ topic }} to a {{ audience }} in two sentences.") .with_context_value("topic", "ownership") .with_context_value("audience", "Python developer"); let rendered = prompt.render()?; let answer = openai.prompt(prompt).await?; ``` A `Prompt` is the text you send to a model, plus the values to fill into it. `Prompt::from` takes a `&'static str` or a `String`. The text is a [Tera](https://keats.github.io/tera/docs/) template, so `{{ topic }}` is a placeholder. `with_context_value` adds one value to the template context. It accepts anything that converts into a `serde_json::Value`, including nested objects from `serde_json::json!`. `render` returns the final text, or an error if the template does not render. The snippet calls it only to show the result. `SimplePrompt::prompt` takes the `Prompt` by value and renders it inside the client. ## Rendering A one-off prompt from `Prompt::from` without template context is sent verbatim. `Prompt::from("hello {{ name }}")` renders to `hello {{ name }}`, braces included. A named template always renders through Tera, so a placeholder without a value fails there. Once you set any value, Tera renders the whole text. Autoescaping is off for one-off templates, so characters such as `<` and `&` reach the model unchanged. A render failure is an `anyhow::Error` from `render`. Clients return it as a `LanguageModelError::PermanentError`, so a retry does not help. `with_context_value` consumes the prompt and returns it. When you hold a `&mut Prompt`, use `insert_context_value` instead. A struct does not convert into a `tera::Context`. Pass it through `tera::Context::from_serialize(&value)?` and hand the result to `with_context`. ## Template syntax Tera supports loops, conditions and its built-in filters in any prompt. ```rust let prompt = Prompt::from("Review these files:\n{% for file in files %}- {{ file }}\n{% endfor %}") .with_context_value("files", vec!["src/lib.rs", "src/main.rs"]); ``` The Tera documentation lists the full syntax. Swiftide builds Tera without its `builtins` feature. The `date`, `slugify`, `filesizeformat`, `urlencode` and `urlencode_strict` filters are missing, as are the `now` and `get_random` functions. Enable `builtins` on your own `tera` dependency if you need them. ## Named templates A long prompt that you use in several places belongs in a named template. You compile it once and load it by name. ```rust let mut templates = tera::Tera::default(); templates.add_raw_template( "review", "Review this {{ language }} diff and list risks:\n{{ diff }}", )?; Prompt::extend(&templates)?; let prompt = Prompt::from_compiled_template("review") .with_context_value("language", "Rust") .with_context_value("diff", "- let x = 1;\n+ let x = 2;"); ``` `Prompt::extend` merges your Tera instance into one repository that the whole process shares. Its templates, filters and functions carry over. `Prompt::from_compiled_template` stores only the name. The lookup happens when the prompt renders, so an unknown name fails at that point. Swiftide does not re-export Tera. Add it to your crate to build the instance: ```sh cargo add tera@1.20 --no-default-features ``` Tera keeps the first template it sees under a name. Calling `Prompt::extend` again with a template of the same name does not replace the registered one. Tera also autoescapes named templates whose name ends in `.html`, `.htm` or `.xml`, so give prompts another suffix. ## Transformer prompts Every indexing transformer that prompts a model has a `prompt_template` field. You replace its default prompt on the builder. ```rust let qa = MetadataQAText::builder() .client(openai) .num_questions(3_usize) .prompt_template( "Write {{ questions }} questions a support engineer would ask about:\n\ {{ node.chunk }}", ) .build()?; ``` The transformer fills the template context before each call. It adds the current node as `node`, so `{{ node.chunk }}` is the chunk text. It also adds its own values. `MetadataQAText` adds `questions`, which is `num_questions` and defaults to 5. The same field exists on `MetadataSummary`, `MetadataTitle` and `MetadataKeywords`, and on `MetadataQACode` and `CompressCodeOutline` behind the `tree-sitter` feature flag. Their default templates are `.prompt.md` files in the Swiftide source. Read the default for a transformer to see which variables it uses. [Transformers and metadata](/indexing/transformers-and-metadata/) covers the transformers themselves. ## Answer prompts The answer generator `answers::Simple` takes a `prompt_template` too. ```rust let answer = answers::Simple::builder() .client(openai) .prompt_template(Prompt::from( "Answer {{ question }} using only:\n{{ documents }}", )) .build()?; ``` Its template context has four variables: * `question`: the question the user asked. * `original`: the same as `question`. * `current`: the query text after the query transformers ran. After a response transformer such as `Summary`, it holds that transformer’s output instead. * `documents`: the retrieved documents, joined with `---` lines. After a response transformer, it holds the same text as `current`. The optional `document_template` renders each document on its own before the join, with its `content` and `metadata` in the template context. The answer generator skips it when a response transformer ran. The query transformer `GenerateSubquestions` and the response transformer `Summary` also take a `prompt_template`. [Answering](/querying/answering/) shows them in a query pipeline. ## Agent system prompts An agent’s system prompt is a `Prompt`, so it takes template values like any other prompt. ```rust let agent = Agent::builder() .llm(&openai) .system_prompt( Prompt::from("You review {{ language }} code. Be brief.") .with_context_value("language", "Rust"), ) .build()?; ``` `system_prompt` accepts anything that converts into a `Prompt`, so a plain string works too. An agent has no system prompt unless you set one. The system prompt renders once, when the agent first runs, after the `BeforeAll` hooks. A `BeforeAll` hook can call `system_prompt_mut` and `insert_context_value` to fill a value just in time. Later hooks run after the render, so a change there does not reach the system message. If rendering fails, the run returns `AgentError::FailedToRenderSystemPrompt`. [System prompts](/agents/system-prompts/) covers the agent side. ## Gotchas * A borrowed `&str` that is not `'static` does not convert into a `Prompt`. Turn it into a `String` with `to_string` first. * `format!` treats `{{` and `}}` as escaped braces. `format!("Explain {{ topic }}")` yields `Explain { topic }`, which is no longer a placeholder. Write `{{{{ topic }}}}` inside `format!`. * Filters and functions registered through `Prompt::extend` work only in named templates. A one-off prompt from `Prompt::from` renders with a fresh Tera instance and does not see them. * `answers::Simple::builder().prompt_template` takes a `Prompt`, not a string. Wrap the text in `Prompt::from`. * The prompt module rustdoc mentions a Template type. It does not exist at this version, so use `Prompt`. ## Reference * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`MetadataQAText`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQAText.html) * [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQACode.html) * [`CompressCodeOutline`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.CompressCodeOutline.html) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) * [`GenerateSubquestions`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.GenerateSubquestions.html) * [`response_transformers::Summary`](https://docs.rs/swiftide/0.34/swiftide/query/response_transformers/struct.Summary.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) ## Next [Structured output](/models/structured-output/) # Structured output > Ask a model for a Rust type with structured_prompt::, where T derives schemars JsonSchema, and use DynStructuredPrompt when you need a trait object. By the end of this page, a model call returns your own Rust struct. The provider shapes its answer to the struct’s JavaScript Object Notation (JSON) Schema. Bring the types and traits into scope: ```rust use schemars::{JsonSchema, schema_for}; use serde::Deserialize; use swiftide::{ agents::{Agent, StopReason, tools::control::StopWithArgs}, integrations::openai::OpenAI, traits::{DynStructuredPrompt, StructuredPrompt as _}, }; ``` Describe the answer as a type: ```rust /// What a model reads out of a changelog. #[derive(Debug, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] pub struct ReleaseSummary { /// One line that names the release theme. pub title: String, /// Changes a user notices. pub highlights: Vec, /// True when upgrading needs code changes. pub breaking: bool, } ``` Then ask the model for it: ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .build()?; let summary = openai .structured_prompt::( format!("Summarise this changelog:\n{changelog}").into(), ) .await?; println!("{}: {} highlights", summary.title, summary.highlights.len()); if summary.breaking { println!("This release needs code changes."); } ``` ## Schema and deserialization The doc comments on the fields become descriptions in the schema, so the model reads them as instructions. `structured_prompt::` is a method of the `StructuredPrompt` trait, so the trait must be in scope. The method takes a `Prompt`, so `.into()` turns a `String` or a `&'static str` into one. Templates and template context work as on [Prompts and templates](/models/prompts-and-templates/). Swiftide builds the schema with `schemars::schema_for!(T)` and sends it with the rendered prompt. It then deserializes the JSON the model returns into `T`. When the JSON does not fit `T`, the call returns `LanguageModelError::PermanentError`. You need schemars, serde and serde_json in your own crate: ```sh cargo add swiftide@0.34 --features openai cargo add schemars@1 serde serde_json --features serde/derive ``` `structured_prompt` itself needs no extra feature flag. The `json-schema` feature only derives `JsonSchema` on some of Swiftide’s own types. ### Schemas for Swiftide types With `json-schema`, `ToolCall`, `ToolSpec`, `ToolFeedback`, `StopReason` and the agent `State` derive `JsonSchema`. Your own types can then hold them as fields and still derive a schema, for example the request body of an approval endpoint: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents,json-schema ``` ```rust /// The body your approval endpoint accepts; the `json-schema` feature flag lets it derive /// `JsonSchema` with Swiftide types as fields. #[derive(Debug, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] pub struct ApprovalDecision { /// The call the agent stopped on, as `StopReason::FeedbackRequired` reported it. pub tool_call: swiftide::chat_completion::ToolCall, /// The person's decision. pub feedback: swiftide::traits::ToolFeedback, } /// Prints the JSON Schema of the approval body, for example for an OpenAPI document. pub fn approval_schema() -> anyhow::Result<()> { let schema = schema_for!(ApprovalDecision); println!("{}", serde_json::to_string_pretty(&schema)?); Ok(()) } ``` `StopReason` and `State` live in `swiftide::agents`, so they also need the `swiftide-agents` feature flag. The `json-schema` feature adds no API of its own. ## Schema rules OpenAI clients send the schema in strict mode. This holds for Chat Completions and for the Responses application programming interface (API). Strict mode has two rules for every object in the schema. The first rule is `additionalProperties: false`. Add `#[serde(deny_unknown_fields)]` to the struct, and to every struct it contains. Schemars then writes `additionalProperties: false` for that object. Without it, OpenAI rejects the request with an error about `additionalProperties`. The second rule is that every field is listed in `required`. Schemars leaves `Option` fields and `#[serde(default)]` fields out of `required`, so OpenAI rejects them. Use a required field with a sentinel value, or an enum, instead. Swiftide passes the schemars schema through as it is. It does not rewrite the schema to fit a provider, so the provider’s rules on schemas apply to your type directly. ## Trait objects `StructuredPrompt` is generic over `T`, so it is not dyn safe. You cannot store a `Box`. Use `DynStructuredPrompt` instead: ```rust let llm: Box = Box::new(openai); let value = llm .structured_prompt_dyn( "Summarise the 0.33 release.".into(), schema_for!(ReleaseSummary), ) .await?; let summary: ReleaseSummary = serde_json::from_value(value)?; ``` `structured_prompt_dyn` takes a `schemars::Schema` and returns a `serde_json::Value`. You deserialize the value yourself. Every `DynStructuredPrompt` gets `StructuredPrompt` through a blanket implementation. To add structured output to your own client, implement `DynStructuredPrompt` only. ## Provider support | Client | Feature flag | How the schema is sent | | ----------------------------------------------------- | ------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------- | | `OpenAI` | `openai` | Chat Completions `response_format` with a JSON Schema. With `use_responses_api(true)`, the Responses API text format. | | `Ollama`, `Groq`, `Gemini`, `OpenRouter`, `Dashscope` | `ollama`, `groq`, `gemini`, `open-router`, `dashscope` | Same as `OpenAI`. | | `AwsBedrock` | `aws-bedrock` | Converse `output_config` with a JSON Schema. | | `Anthropic` | `anthropic` | Not supported. | | `FastEmbed` | `fastembed` | Not supported. | Whether a JSON Schema response format works on the `GenericOpenAI` aliases depends on the endpoint and the model. A structured call is not retried. [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/) explains why. ## Agents An agent talks to its model through chat completion, not through `structured_prompt`. To get a typed final answer from an agent, replace the default stop tool with `StopWithArgs`. Agents live behind the `swiftide-agents` feature flag: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents ``` ```rust let mut agent = Agent::builder() .llm(&openai) .without_default_stop_tool() .tools([StopWithArgs::with_parameters_schema(schema_for!( ReleaseSummary ))]) .build()?; agent.query("Summarise the 0.33 release.").await?; let summary: Option = match agent.stop_reason() { Some(StopReason::RequestedByTool(_, Some(payload))) => { Some(serde_json::from_value(payload.clone())?) } _ => None, }; ``` The model ends the run by calling `stop` with arguments that follow the `ReleaseSummary` schema. The stop reason is then `StopReason::RequestedByTool(_, Some(payload))`, where `payload` is the arguments as a `serde_json::Value`. `serde_json::from_value` turns it into your type. The stop tool’s schema follows different rules than `structured_prompt`. Swiftide normalizes tool schemas for OpenAI: it fills in `required` and adds `additionalProperties: false`. [Stopping and structured results](/agents/stopping-and-structured-results/) covers stop tools. Tool arguments use schemars schemas too, so the same derive works for a tool’s arguments; see [Tools](/agents/tools/). ## Reference * [`StructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.StructuredPrompt.html) * [`DynStructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.DynStructuredPrompt.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`GenericOpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html) * [`AwsBedrock`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrock.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`StopWithArgs`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.StopWithArgs.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`LanguageModelError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.LanguageModelError.html) ## Next [Images and multimodal input](/models/images/) # Generating answers > Turn retrieved documents into an answer with answers::Simple, a custom prompt and a document template that lets the answer cite the file each statement comes from. The last step of a query pipeline sends the question and the retrieved documents to a large language model (LLM). Here you set that step up so every answer names the file it came from, the way the [ask-your-codebase](/showcases/ask-your-codebase/) showcase does. `answers::Simple` needs no feature flag. The snippets below use the OpenAI client and Qdrant, which sit behind the `openai` and `qdrant` feature flags: ```sh cargo add swiftide@0.34 --features openai,qdrant ``` examples/ask-your-codebase/src/query.rs ```rust let search = SimilaritySingleEmbedding::default().with_top_k(top_k); let pipeline = query::Pipeline::from_search_strategy(search) // Rephrase the question into several, to widen what the search can match .then_transform_query( GenerateSubquestions::builder() .client(llm.clone()) .prompt_template(SUBQUESTIONS_PROMPT.into()) .build()?, ) // Embed with the same local model that embedded the chunks .then_transform_query(Embed::from_client(embedding::local_model()?)) .then_retrieve(codebase.vector_store()?) // Answer the original question from the retrieved code, each chunk under its file path .then_answer( answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(CHUNK_TEMPLATE) .build()?, ); let answered = pipeline.query(question).await?; ``` ## The answer generator `then_answer` takes an answer generator and moves the query from `Retrieved` to `Answered`. Swiftide ships one: `answers::Simple`. It renders a prompt, sends it through any client that implements `SimplePrompt`, and stores the reply as the answer. `answers::Simple::from_client(llm)` builds it with the default prompt. The builder lets you replace two templates: * `prompt_template` is the whole prompt. * `document_template` renders each retrieved document before they are joined. It is optional. Without a custom prompt, the model receives this text, shown here with `What is love?` as the question and `My context` as the documents: ```text Answer the following question based on the context provided: What is love? ## Constraints * Do not include any information that is not in the provided context. * If the question cannot be answered by the provided context, state that it cannot be answered. * Answer the question completely and format it as markdown. ## Context --- My context --- ``` The prompt template sees four variables: * `question`: the question the user asked, from `Query::original`. * `original`: the same text as `question`. * `current`: the current query, from `Query::current`. * `documents`: the retrieved documents as one string, or the text of a response transformer that changed `current` (see below). Prompts are Tera templates; [Prompts and templates](/models/prompts-and-templates/) covers the syntax. `original` and `current` need 0.33 or later; see [Upgrading](/reference/upgrading/). ## Source citations The answer can only name a file if the prompt contains the file name. The document template puts it there. examples/ask-your-codebase/src/query.rs ````rust /// Widens the question into the follow-up questions its answer involves. /// /// They are phrased like the questions generated for each chunk while indexing, so the two /// embed close together. const SUBQUESTIONS_PROMPT: &str = r" Your job is to help a search find the code that answers a developer's question about a codebase. Question: {{ question }} Write {{ num_questions }} short follow-up questions that the code answering this question would also answer. Think about the mechanisms involved: the main loop, the types and enum variants, the conditions that are checked, the functions that are called, and the configuration that changes the outcome. ## Constraints * Phrase them the way a developer would ask about this codebase, e.g. 'When does the agent stop?'. * Do not guess names of types or functions. * Respond with the original question and the follow-up questions only, one per line, prefixed with '- '. "; /// Renders a retrieved chunk under its file path, so the answer can cite it. /// /// `metadata.path` is the `PATH_FIELD` metadata; a test keeps the two in step. const CHUNK_TEMPLATE: &str = r"### {{ metadata.path }} ``` {{ content }} ``` "; /// Answers from the retrieved code and cites files. const ANSWER_PROMPT: &str = r" Answer a developer's question about a codebase, using snippets of its source code. Question: {{ question }} ## Constraints * Only use the snippets below. If they do not answer the question, say briefly what is missing. * Be concrete: name the types, functions and variants involved and explain when they apply. * Describe behaviour: what calls what, in which order, and under which conditions. * Cite the file each statement comes from in backticks, e.g. (`src/lib.rs`). * Use concise markdown. Add a short code example only if it shows how to use the API. ## Snippets {{ documents }} "; ```` `CHUNK_TEMPLATE` renders one document. Its template context is the serialized `Document`, so `content` is the chunk text and `metadata` holds the metadata fields your storage kept. ask-your-codebase writes more metadata while indexing, but its LanceDB table keeps only `path`, so `{{ metadata.path }}` becomes a heading above each chunk. `answers::Simple` joins the rendered documents with `---` lines and inserts the result as `{{ documents }}`. `ANSWER_PROMPT` then asks for a citation after each statement. With `claude-sonnet-5-5`, the answers follow it: ```text $ ./target/debug/ask-your-codebase ask --model claude-sonnet-5-5 --path swiftide-agents "How does an agent decide to stop?" # How an agent decides to stop An agent stops when something calls `Agent::stop` (or `stop_with_error`), which moves the state to `State::Stopped(StopReason)`. The main loop in `run_agent` checks that state after each step. `Agent::stop` is a no-op if the agent is already stopped (`src/agent.rs`). ## The stop mechanism - `Agent::stop(reason)` fires the `OnStop` hooks with the reason, then sets `self.state = State::Stopped(reason)`. It only takes effect at the end of the current loop iteration (`src/agent.rs`). ... Sources: - src/agent.rs - src/tools/control.rs - src/lib.rs ... ``` The `Sources:` list does not come from the model. The answered query still holds the documents it was answered from, so the showcase reads their paths back: examples/ask-your-codebase/src/query.rs ```rust impl From> for Answer { /// Keeps the distinct file paths of the retrieved chunks, most relevant first. fn from(query: Query) -> Self { let mut sources: Vec = Vec::new(); for document in query.documents() { let Some(path) = document.metadata().get(PATH_FIELD).and_then(|v| v.as_str()) else { continue; }; if !sources.iter().any(|source| source == path) { sources.push(path.to_owned()); } } Self { text: query.answer().to_owned(), sources, } } } ``` `answer` returns the text and `documents` returns the documents the answer was built from. A response transformer can reorder or trim that list first: the [docs-qa](/showcases/docs-qa/) showcase reranks its documents, so its numbered list of page and section titles comes best first. ## Original and current question A query transformer such as `GenerateSubquestions` rewrites the current query. `question` in the prompt stays the user’s question, so the model answers what was asked, not the list of subquestions. Use `current` when the model should also see what the search looked for: ```rust let answerer = answers::Simple::builder() .client(openai) .prompt_template(Prompt::from( "Answer this question: {{ original }}\n\ The search looked for: {{ current }}\n\ Cite the file of every statement in backticks.\n\n\ {{ documents }}", )) .document_template("File: {{ metadata.path }}\n{{ content }}") .build()?; ``` `answers::Simple` decides what goes into `documents` like this: * If a response transformer changed `current` after retrieval, `documents` is that text and the document template is skipped. `response_transformers::Summary` works this way; see [Reranking and response transformers](/querying/reranking-and-response-transformers/). * Otherwise each document is rendered with the document template, or as its plain content without one. The check compares `current` with the query recorded by the `Retrieved` event, so a rewritten query from before retrieval does not count as a change. A reranker that only reorders documents leaves `current` alone, so the document template still applies. [Transforming queries](/querying/transforming-queries/) explains the transformation history. ## Custom answer generators Implement the `Answer` trait to control the last step yourself. This one skips the LLM call when the retriever found nothing: ```rust /// Answers with the LLM only when the retriever found something. #[derive(Clone)] pub struct AnswerWhenFound { inner: answers::Simple, } #[async_trait] impl Answer for AnswerWhenFound { async fn answer( &self, query: Query, ) -> anyhow::Result> { if query.documents().is_empty() { return Ok(query.answered("No matching documents were found.")); } self.inner.answer(query).await } } ``` `Query::answered` stores the text and moves the query to `Answered`. The trait requires `Clone`, so derive it. Every other case goes to the wrapped `answers::Simple`. The generator goes where `answers::Simple` went: ```rust let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_answer(AnswerWhenFound { inner: answers::Simple::from_client(openai), }); let answered = pipeline.query("How do I configure retries?").await?; println!("{}", answered.answer()); ``` `then_answer` also accepts a synchronous `Fn(Query) -> Result> + Send + Sync + Clone` closure, for example a fixed answer in a test. A closure that captures a value which is not `Clone` does not compile. ## Gotchas * `prompt_template` takes a `Prompt`, so write `.into()` or `Prompt::from`. `document_template` converts for you, so a `&'static str` works directly. * A document template that reads a missing metadata key fails to render, and the query returns that error. The [docs-qa](/showcases/docs-qa/) templates guard each key with Tera’s `default` filter, as in `{{ metadata.Title | default(value="") }}`. * After answering, `current` holds the answer; see [Transforming queries](/querying/transforming-queries/) for reading the rewritten query from `history`. * The model can only cite files among the retrieved documents. A chunk outside the top results is invisible to it. Raise `top_k` for broad questions; see [Retrieval and search strategies](/querying/retrieval-and-search-strategies/). * `answers::Simple` makes one LLM call and returns its error. Wrap the client in `LanguageModelWithBackOff` to retry transient failures, as docs-qa does; see [Errors, backoff and usage](/models/errors-backoff-and-usage/). ## Reference * [`query::Pipeline::then_answer`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html#method.then_answer) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) * [`Answer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Answer.html) * [`Query`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Query.html) * [`Document`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Document.html) * [`query_transformers::GenerateSubquestions`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.GenerateSubquestions.html) * [`TransformationEvent`](https://docs.rs/swiftide/0.34/swiftide/query/enum.TransformationEvent.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`response_transformers::Summary`](https://docs.rs/swiftide/0.34/swiftide/query/response_transformers/struct.Summary.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) ## Next [Evaluating with Ragas](/querying/evaluation/) # Evaluating with Ragas > Record each question's retrieved documents and answer with the Ragas evaluator, and export them with ground truths as a JSON dataset for the Python ragas toolkit. You end with a JSON file that holds every question, the answer your query pipeline gave, the documents it retrieved and the answer you expected. You can score that file and compare it between runs. The scores at the end come from the evaluation in the [docs-qa](/showcases/docs-qa/) showcase. ## Evaluation run `Ragas` needs no feature flag. The snippets below use the OpenAI client and Qdrant, which sit behind the `openai` and `qdrant` feature flags: ```sh cargo add swiftide@0.34 --features openai,qdrant ``` ```rust let ragas = Ragas::from_prepared_questions(vec![ ( "What does a retriever return?".to_string(), "A retriever returns documents that match the query.".to_string(), ), ( "Which step turns retrieved documents into an answer?".to_string(), "The answer generator, for example answers::Simple.".to_string(), ), ]); let pipeline = query::Pipeline::default() // Attach the evaluator while the pipeline is still pending .evaluate_with(ragas.clone()) .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(openai)); pipeline.query_all(ragas.questions().await).await?; std::fs::write("ragas.json", ragas.to_json().await)?; ``` `Ragas` is an evaluator: a type that implements `EvaluateQuery`. It holds the dataset, and its clones share that dataset. Pass one clone to the pipeline with `evaluate_with` and keep the other to read the results. `ragas.questions()` returns one pending query per question in the dataset, and `query_all` runs them all. The pipeline calls the evaluator twice per question. Right after `then_retrieve`, `Ragas` stores the content of the retrieved documents as the contexts of that question. After `then_answer`, it stores the answer. `to_json` then writes the dataset. ### Evaluation in docs-qa The [docs-qa](/showcases/docs-qa/) showcase runs the same steps on its own question set: examples/docs-qa/src/eval.rs ```rust let ragas = Ragas::from_prepared_questions(&questions); let answered = qa .evaluated_pipeline(ragas.clone()) .query_all(ragas.questions().await) .await?; let ragas_json = ragas.to_json().await; ``` `evaluated_pipeline` builds the same pipeline as the `ask` command and adds `evaluate_with(ragas)` to it. docs-qa also enables the `fastembed` feature flag for its hybrid search. ## Questions and ground truths `Ragas::from_prepared_questions` takes anything that converts into an `EvaluationDataSet`. An `EvaluationDataSet` converts from: * `Vec` or `&[String]`: questions only, with an empty ground truth. * `Vec<(String, String)>`: pairs of question and ground truth, as in the first snippet. * a JSON string, through `parse`, in the format that `to_json` writes. docs-qa passes `&QuestionSet`, its own YAML-backed type, which implements `From<&QuestionSet> for EvaluationDataSet`. Implement `From` for your own type in the same way. A ground truth is the answer you expect, written in your own words. Scores that compare the answer with the ground truth need it, so write one for every question you can. ### Baseline ground truths Writing ground truths by hand takes time. You can let a first run write them and review them afterwards. ```rust // Runs the pipeline once over every question in the dataset let evaluate = async |ragas: &Ragas| -> anyhow::Result<()> { query::Pipeline::default() .evaluate_with(ragas.clone()) .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant.clone()) .then_answer(answers::Simple::from_client(openai.clone())) .query_all(ragas.questions().await) .await?; Ok(()) }; // First run: questions only, then keep this run's answers as the ground truth let questions = vec![ "What does a retriever return?".to_string(), "Which step turns retrieved documents into an answer?".to_string(), ]; let ragas = Ragas::from_prepared_questions(questions); evaluate(&ragas).await?; ragas.record_answers_as_ground_truth().await; std::fs::write("baseline.json", ragas.to_json().await)?; // Later runs: load the reviewed baseline and evaluate against it let dataset: EvaluationDataSet = std::fs::read_to_string("baseline.json")?.parse()?; let ragas = Ragas::from_prepared_questions(dataset); evaluate(&ragas).await?; ``` The `evaluate` closure builds a pipeline with `evaluate_with(ragas.clone())` and runs `query_all` over `ragas.questions()`, so each call fills the dataset with answers and contexts. `record_answers_as_ground_truth` then copies each recorded answer into its ground truth. Read the file and fix wrong answers before you treat it as the baseline. Parsing an export keeps its ground truths. The next run overwrites the answers and contexts it loaded. ## The dataset The export is a pretty-printed JSON array with one object per question. This is the output that the unit test for `to_json` expects for one question with a ground truth that has not run yet: ```json [ { "answer": "", "contexts": [], "ground_truth": "A programming language", "question": "What is Rust?" } ] ``` The keys are in alphabetical order, unless a crate in your build enables the serde_json feature that preserves insertion order. The rows are in hash map order, which changes from run to run. docs-qa sorts the rows back into the order of its questions file, so the exports of two runs can be diffed. `contexts` holds every retrieved document, recorded before any response transformer runs. docs-qa retrieves 20 candidates and reranks them down to 10, so its `ragas.json` has 20 contexts per question, while the answer generator saw 10. Ragas metrics computed from `contexts` therefore score the retriever’s 20 candidates, not the 10 documents the answer generator saw. ## Scores `Ragas` computes no scores. The JSON is input for the Python [ragas](https://docs.ragas.io) toolkit, which calculates metrics such as faithfulness and context recall. docs-qa shows numbers without Python. Its judge asks a large language model (LLM) to split the ground truth and the answer into claims and to check each one, using [structured output](/models/structured-output/). Its scores are ratios modelled on the ragas metrics of the same names. In this run `claude-sonnet-5-5` both answered and judged: ```sh ./target/debug/docs-qa --model claude-sonnet-5-5 eval --judge-model claude-sonnet-5-5 \ --out examples/docs-qa/ragas.json ``` ```text # question ctx faithful answer context recall recall 1 Which crate features and environment variable do I … 10 1.00 1.00 1.00 2 What does the agents-contrib-context feature provid… 10 1.00 1.00 1.00 3 What are the default limits of the read_file tool? 10 1.00 1.00 1.00 4 Which storage and retrieval backends does Swiftide … 10 1.00 1.00 1.00 5 What should I run before opening a pull request? 10 1.00 1.00 1.00 6 What is a task in Swiftide? 10 1.00 1.00 1.00 7 When does ContextCompression summarize the conversa… 10 1.00 1.00 1.00 8 Which commit message convention does the project fo… 10 1.00 1.00 1.00 9 How do I add a custom tool to an agent with the too… 10 1.00 1.00 1.00 mean 1.00 1.00 1.00 RAGAS dataset written to examples/docs-qa/ragas.json ``` * `ctx` is the number of chunks the LLM saw after reranking. * `faithful` is the share of the answer’s claims that those chunks support. * `answer recall` is the share of the ground truth’s claims that the answer states. * `context recall` is the share of the ground truth’s claims that those chunks support. Before retrieval was tuned, two questions scored lower. One fix added page and title to the rerank template; see [Rerank templates](/querying/reranking-and-response-transformers/#rerank-templates). The other changed the indexing prompt so code-only chunks get titles that name the API they show; see the [docs-qa showcase](/showcases/docs-qa/). ## Custom evaluators Any type that implements `EvaluateQuery` can take the place of `Ragas`. The trait has one async method, `evaluate`, which receives a `QueryEvaluation`: `RetrieveDocuments` with the query after retrieval, or `AnswerQuery` with the answered query. The trait requires `Clone`, so keep shared state behind an `Arc`. This evaluator collects the questions for which the retriever found no documents: ```rust /// Flags questions for which the retriever found nothing. #[derive(Clone, Default)] pub struct EmptyRetrievals { questions: Arc>>, } #[async_trait] impl EvaluateQuery for EmptyRetrievals { async fn evaluate(&self, evaluation: QueryEvaluation) -> anyhow::Result<()> { if let QueryEvaluation::RetrieveDocuments(query) = evaluation && query.documents().is_empty() { self.questions .lock() .unwrap() .push(query.original().to_string()); } Ok(()) } } ``` ```rust let empty = EmptyRetrievals::default(); query::Pipeline::default() .evaluate_with(empty.clone()) .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(openai)) .query("What does a retriever return?") .await?; for question in empty.questions.lock().unwrap().iter() { println!("no documents for: {question}"); } ``` An error from `evaluate` fails the query, so return `Ok` for anything that should only be recorded. ## Gotchas * `evaluate_with` exists only on a pending pipeline. Call it before `then_retrieve`; its position among query transformers does not matter. * `Ragas` matches each query to its row by `original()`. A query whose question is not in the dataset fails with `Question not found`, and so does `query_all`. Query transformers change `current()`, not `original()`, so they do not affect the match. * Rows are keyed by question text, so each question must be unique. docs-qa rejects a questions file with duplicates. * `query_all` returns answers in the order they finish; see [Query methods](/querying/overview/#query-methods). docs-qa matches each answer back to its question with `original()`. * Without the docs-qa workaround in [Retrieval and search strategies](/querying/retrieval-and-search-strategies/), Qdrant contexts in the export arrive `Debug`-quoted in 0.33. * Nine questions and one LLM judge make a regression signal for one corpus, not a benchmark. Keep the question set and the judge model fixed when you compare runs. ## Reference * [`Ragas`](https://docs.rs/swiftide/0.34/swiftide/query/evaluators/ragas/struct.Ragas.html) * [`EvaluationDataSet`](https://docs.rs/swiftide/0.34/swiftide/query/evaluators/ragas/struct.EvaluationDataSet.html) * [`EvaluateQuery`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EvaluateQuery.html) * [`QueryEvaluation`](https://docs.rs/swiftide/0.34/swiftide/query/enum.QueryEvaluation.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`Query`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Query.html) * [docs-qa source on GitHub](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa) ## Next [Models](/models/overview/) # How querying works > A query pipeline takes a question from Pending to Retrieved to Answered through query transformers, a retriever, response transformers and an answer generator. You end up with a query pipeline that answers questions about your own code and names the files each answer came from. It is the query side of the [ask-your-codebase](/showcases/ask-your-codebase/) showcase, a retrieval augmented generation (RAG) command line tool. The pipeline reads the `code_chunks` LanceDB table that the showcase’s [indexing pipeline](/indexing/overview/) fills with the same embedding model, so run `ask-your-codebase index` first. The full program needs the `openai`, `fastembed` and `lancedb` feature flags. The helpers and prompt constants live in the showcase. ## The ask-your-codebase pipeline examples/ask-your-codebase/src/query.rs ```rust let search = SimilaritySingleEmbedding::default().with_top_k(top_k); let pipeline = query::Pipeline::from_search_strategy(search) // Rephrase the question into several, to widen what the search can match .then_transform_query( GenerateSubquestions::builder() .client(llm.clone()) .prompt_template(SUBQUESTIONS_PROMPT.into()) .build()?, ) // Embed with the same local model that embedded the chunks .then_transform_query(Embed::from_client(embedding::local_model()?)) .then_retrieve(codebase.vector_store()?) // Answer the original question from the retrieved code, each chunk under its file path .then_answer( answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(CHUNK_TEMPLATE) .build()?, ); let answered = pipeline.query(question).await?; ``` The pipeline runs four steps for each question: 1. `GenerateSubquestions` asks the large language model (LLM) to rewrite the question into several related questions. Their text becomes the current query. 2. `Embed` embeds the current query with the local FastEmbed model that also embedded the chunks during indexing. 3. `then_retrieve` hands the query to the LanceDB storage, the retriever. It returns the `top_k` most similar chunks as documents. 4. `answers::Simple`, the answer generator, renders the documents into a prompt and asks the LLM for the answer. `SimilaritySingleEmbedding` is the search strategy. [Retrieval and search strategies](/querying/retrieval-and-search-strategies/) explains the `let mut` binding and its type annotation. This is the start of a real run against the `swiftide-agents` crate. The answer came from `claude-sonnet-5-5`, and the embeddings from `jina-embeddings-v2-base-code` running locally: ```text $ ./target/debug/ask-your-codebase ask --model claude-sonnet-5-5 --path swiftide-agents "How does an agent decide to stop?" # How an agent decides to stop An agent stops when something calls `Agent::stop` (or `stop_with_error`), which moves the state to `State::Stopped(StopReason)`. The main loop in `run_agent` checks that state after each step. `Agent::stop` is a no-op if the agent is already stopped (`src/agent.rs`). ... Sources: - src/agent.rs - src/tools/control.rs - src/lib.rs - src/state.rs - src/errors.rs - src/task_agent.rs - src/hooks.rs - src/tasks.rs - src/default_context.rs ``` The sources list comes from the `path` metadata of the documents in the answered query. [Generating answers](/querying/answering/) shows the showcase’s `sources` code and the document template that puts the file path in front of each chunk. ## Query states ![Indexing and query pipelines](/diagrams/diagram-pipeline.svg) A question travels through the pipeline as a `Query`. The state is a type parameter, and it moves from `states::Pending` to `states::Retrieved` to `states::Answered`. A query keeps the original question in `original()` and the text of the latest step in `current()`. A query transformer sets `current()` to the rewritten question. A response transformer such as `Summary` can replace it with a summary, and `answers::Simple` then uses that text in place of the documents. The retriever adds documents, which you read with `documents()`. Once answered, `current()` holds the answer, and `answer()` returns it. The pipeline carries the same state, with different methods per state: | Pipeline state | Methods | | ------------------- | -------------------------------------------------------- | | `states::Pending` | `then_transform_query`, `then_retrieve`, `evaluate_with` | | `states::Retrieved` | `then_transform_response`, `then_answer` | | `states::Answered` | `query`, `query_mut`, `query_all` | The compiler enforces this order. A pipeline without `then_retrieve` has no `then_answer`, and a pipeline without `then_answer` has no `query`. ## Pipeline steps Each step method takes one trait: | Step | Trait | Called | | ------------------------- | ---------------------------------- | ------------------------------------- | | `then_transform_query` | `TransformQuery` | any number of times, before retrieval | | `then_retrieve` | `Retrieve` for the search strategy | once | | `then_transform_response` | `TransformResponse` | any number of times, after retrieval | | `then_answer` | `Answer` | once | Every step accepts a struct that implements the trait, a `Box` or `Arc` of the trait, or a closure. Closures are synchronous and must be `Clone + Send + Sync`. Each step runs every query on its own Tokio task with `tokio::spawn`, so the pipeline needs a Tokio runtime. ### Closure steps ```rust let pipeline = query::Pipeline::default() .then_transform_query(|mut query: Query| { let lowercased = query.current().to_lowercase(); query.transformed_query(lowercased); Ok(query) }) .then_retrieve( |_: &SimilaritySingleEmbedding, query: Query| { let documents = vec![Document::from( "Swiftide is a Rust library for LLM applications.", )]; Ok(query.retrieved_documents(documents)) }, ) .then_transform_response(Ok) .then_answer(|query: Query| { let answer = format!("Found {} document(s)", query.documents().len()); Ok(query.answered(answer)) }); let answered = pipeline.query("What is Swiftide?").await?; println!("{}", answered.answer()); ``` This pipeline needs no model and no storage, which makes it a starting point for tests. `retrieved_documents` and `answered` are the only ways to move a `Query` to the next state, and `transformed_query` records each rewrite in `history()`. ## Search strategy `query::Pipeline::default()` uses `SimilaritySingleEmbedding` with a `top_k` of 10 and no filter. Put an `Embed` query transformer before `then_retrieve`. The retriever searches with the embedding it sets. `query::Pipeline::from_search_strategy` starts a pipeline with any other strategy: `SimilaritySingleEmbedding` with a different `top_k` or a filter, `HybridSearch` or `CustomStrategy`. The retriever must implement `Retrieve` for that strategy. If it does not, the pipeline does not compile. [Retrieval and search strategies](/querying/retrieval-and-search-strategies/) lists which storage supports which strategy. ## Query methods ```rust let mut pipeline = query::Pipeline::default() .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant.clone()) .then_answer(answers::Simple::from_client(openai.clone())); let first = pipeline.query_mut("What is a query pipeline?").await?; let second = pipeline.query_mut("What does a retriever return?").await?; println!("{}\n\n{}", first.answer(), second.answer()); let answered = query::Pipeline::default() .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(openai)) .query_all(vec!["What is a node?", "What is a chunk?"]) .await?; for query in answered { println!("{}: {}", query.original(), query.answer()); } ``` `query` consumes the pipeline. `query_mut` borrows the pipeline mutably, so you can call it again for the next question. `query_all` takes a `Vec` of questions, runs them through the steps concurrently and returns every answered query. `query_all` returns results in the order they finish, not in the order you passed them. Match each result to its question with `original()`. An empty `Vec` never returns, because the pipeline waits for an answer that does not come. The first error ends the call, and the answers that already finished are lost. Every query method accepts anything that implements `AsRef`, such as `&str` or `String`. ## Gotchas * The pipeline steps are lazy. No step runs until `query`, and errors from a model or a storage show up as the error of `query`. Code that builds a step can still do work: in the showcase, `embedding::local_model()` loads the FastEmbed model, and downloads it on first use, before the pipeline exists. * `query` and `query_mut` log `Answered query in N seconds` for every question, and `query_mut` also logs `Sending query`. `query_all` logs `Sending queries` and `Answered all queries in N seconds` once per batch. All are at warn level, so filter the `swiftide_query` target in your tracing subscriber if that is too noisy. * Query transformers, response transformers and answer generators must be owned and `'static`. Pass clients by value and `clone()` the ones you use in more than one step. `then_retrieve` also accepts a reference, such as `&qdrant`, and clones it. * A closure that captures a value without `Clone`, such as a lock guard, does not implement the step trait and fails with a trait-bound error. * Each step runs up to one query per logical central processing unit (CPU) at the same time. The pipeline has no setting to change that. * The source of the query pipeline marks it as being in an early stage. Expect its application programming interface (API) to change between releases, and check [Upgrading](/reference/upgrading/) when you bump Swiftide. * The query pipeline needs no feature flag. The models and retrievers do, for example `openai`, `fastembed`, `lancedb` or `qdrant`. See [Feature flags](/reference/feature-flags/). ## Reference * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`Query`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Query.html) * [`states`](https://docs.rs/swiftide/0.34/swiftide/query/states/index.html) * [`Document`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Document.html) * [`TransformQuery`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.TransformQuery.html) * [`Retrieve`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Retrieve.html) * [`TransformResponse`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.TransformResponse.html) * [`Answer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Answer.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`GenerateSubquestions`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.GenerateSubquestions.html) * [`query_transformers::Embed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.Embed.html) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) * [`HybridSearch`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.HybridSearch.html) * [`CustomStrategy`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.CustomStrategy.html) * [`EvaluateQuery`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EvaluateQuery.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) ## Next [Transforming queries](/querying/transforming-queries/) # Reranking and response transformers > Reorder and trim retrieved documents with the FastEmbed Rerank cross-encoder, condense them into an LLM Summary that the answer generator uses instead, or filter them with your own TransformResponse. Your query pipeline fetches twenty candidate documents, keeps the ten a local reranker scores highest, and answers from those. You can also summarize the documents with a large language model (LLM) or filter them with your own response transformer. ## FastEmbed reranker The docs-qa showcase builds its reranker next to its answer generator: examples/docs-qa/src/query.rs ```rust let reranker = Rerank::builder() .top_k(CONTEXT_CHUNKS) .document_template(Some(RERANK_TEMPLATE.to_string())) .build()?; let answerer = answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(DOCUMENT_TEMPLATE) .build()?; ``` It then places the reranker between retrieval and the answer: examples/docs-qa/src/query.rs ```rust pipeline // Hybrid search needs both a dense and a sparse embedding of the question .then_transform_query(query_transformers::Embed::from_client( self.embedders.dense.clone(), )) .then_transform_query(query_transformers::SparseEmbed::from_client( self.embedders.sparse.clone(), )) .then_retrieve(self.qdrant.clone()) // A local cross-encoder reorders the fused candidates; only the best few remain .then_transform_response(self.reranker.clone()) .then_answer(self.answerer.clone()) ``` `Rerank` is a response transformer. A response transformer runs after `then_retrieve` and before `then_answer`. It takes the query in the `Retrieved` state and returns it in the same state. `Rerank` runs a cross-encoder: a model that reads the question and one document together and returns a relevance score. That is slower than comparing embeddings, so you run it only on the candidates the retriever already found. The builder has four fields: * `top_k`: how many documents to keep, as a `usize`. The default is 10. * `document_template`: an optional Tera template that renders each document before scoring. The setter takes an `Option`, so pass `Some(..)`. * `model`: the FastEmbed `TextRerank` model, wrapped in `Arc>`. The default is BGE reranker base. * `model_batch_size`: an optional batch size. The default is FastEmbed’s own. `Rerank` scores every document against `query.original()`, the question as the user asked it. Query transformers such as `GenerateSubquestions` replace `current()`, which the retriever searches with and which `answers::Simple` also sees. The reranker ignores `current()`. It replaces the query’s documents with the `top_k` highest-scoring ones, best first. `Rerank` lives in `swiftide::integrations::fastembed` behind the `fastembed` feature flag: ```sh cargo add swiftide@0.34 --features fastembed ``` This is a docs-qa answer, written by `claude-sonnet-5-5` and reranked locally with BGE reranker base. The sources list the pages and sections of the kept documents, best first: ```text Swiftide supports OpenAI and Azure OpenAI, Anthropic, Gemini, OpenRouter, AWS Bedrock Converse API, Groq, Ollama, and Dashscope. Integrations are feature-gated, so you enable a provider by adding its Cargo feature to `swiftide`. ... Sources: [1] Swiftide › Swiftide capabilities for agents, typed task graphs, pipelines, and tracing (README.md) [2] Swiftide › Swiftide: Rust framework for LLM agents, task graphs, and RAG (README.md) [3] Swiftide › Swiftide Integrations Overview by Feature-Gated Category (README.md) ... [9] Swiftide › Swiftide agent tool defined with the swiftide::tool macro (README.md) [10] Swiftide › Installing Swiftide and Setting Up OpenAI Integration (README.md) ``` The `openai` feature flag in that answer comes from the tenth source, the Quick Start chunk. With a `top_k` below 10 it would not have reached the model. Check a few answers before you lower `top_k`. ### Rerank templates examples/docs-qa/src/query.rs ```rust // The answer prompt receives the question and the rendered documents; each document is rendered // with its page title and the chunk title written during indexing. const ANSWER_PROMPT: &str = "You answer questions about Swiftide, a Rust library for building \ LLM applications, using excerpts from its documentation. Question: {{ question }} ## Instructions * Use only the documentation excerpts below. If they do not answer the question, say so plainly. * Start with the direct answer in one or two sentences, then add supporting details. * Only when the question asks how to do something, include the feature flags, commands or a \ short Rust snippet from the excerpts. Never invent APIs. * Format the answer as markdown and keep it under 250 words. Do not refer to \"the excerpts\". ## Documentation excerpts {{ documents }} "; const DOCUMENT_TEMPLATE: &str = "### {{ metadata.Page | default(value=\"\") }}: \ {{ metadata.Title | default(value=\"\") }} {{ content }}"; // The reranker scores the same title-first rendering. A chunk that is mostly code says little // about what it shows, so its page and the title written during indexing go first. const RERANK_TEMPLATE: &str = "{{ metadata.Page | default(value=\"\") }}: \ {{ metadata.Title | default(value=\"\") }} {{ content }}"; ``` `RERANK_TEMPLATE` puts the page name and the chunk title, both written during indexing, before the content. A chunk that is mostly code says little about its topic, so the extra context helps it score fairly. The template only changes what the model scores. The documents stay as they were, and the answer generator renders them with its own `document_template`. In docs-qa this template fixed one evaluation result. For “What is a task in Swiftide?”, the chunk that lists what a task supports ranked 14th when the reranker scored only chunk text, so it was dropped. The answer missed two features, with answer recall 0.71 and context recall 0.57. With page and title in the template, the chunk made the kept ten and both scores reached 1.00. [Evaluating with Ragas](/querying/evaluation/#scores) explains these scores. Model download, cache location and load failures are covered in [FastEmbed](/integrations/fastembed/). ### Minimal pipeline The single-file example builds its reranker in its setup: ```rust let openai_client = integrations::openai::OpenAI::builder() .default_prompt_model("gpt-4o") .build()?; let fastembed = fastembed::FastEmbed::builder().batch_size(10).build()?; let reranker = fastembed::Rerank::builder().top_k(5).build()?; let qdrant = Qdrant::builder() .batch_size(50) .vector_size(384) .collection_name("swiftide-reranking") .build()?; ``` It then generates subquestions for retrieval, while the reranker still scores against the original question: ```rust // By default the search strategy is SimilaritySingleEmbedding // which takes the latest query, embeds it, and does a similarity search let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::GenerateSubquestions::from_client( openai_client.clone(), )) .then_transform_query(query_transformers::Embed::from_client(fastembed.clone())) .then_retrieve(qdrant.clone()) .then_transform_response(reranker) .then_answer(answers::Simple::from_client(openai_client.clone())); ``` ## Summaries ```rust // By default the search strategy is SimilaritySingleEmbedding // which takes the latest query, embeds it, and does a similarity search let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::GenerateSubquestions::from_client( openai_client.clone(), )) .then_transform_query(query_transformers::Embed::from_client( openai_client.clone(), )) .then_retrieve(qdrant.clone()) .then_transform_response(response_transformers::Summary::from_client( openai_client.clone(), )) .then_answer(answers::Simple::from_client(openai_client.clone())); ``` `response_transformers::Summary` sends the retrieved documents to an LLM and asks for one summary of them. It stores the reply with `transformed_response`, which records a transformation event and replaces `query.current()`. It leaves `documents()` untouched. `answers::Simple` then answers from the summary instead of the rendered documents; see [Generating answers](/querying/answering/). The documents stay on the query, so you can still list sources. To change the instruction, pass your own `prompt_template`: ```rust let summary = response_transformers::Summary::builder() .client(openai) .prompt_template(Prompt::from( "Summarize these excerpts as one markdown list. Keep every type name, \ feature flag and command exactly as written.\n\ {% for document in documents %}\n\ ---\n\ {{ document.metadata.path | default(value=\"unknown\") }}:\n\ {{ document.content }}\n\ {% endfor %}", )) .build()?; ``` The template sees only `documents`, a list of documents with `content` and `metadata`. The question is not in its context. You can chain response transformers. Put `Rerank` before `Summary` and the LLM summarizes only the kept documents. ## Custom response transformers ```rust /// Keeps the first document of each file, in retrieval order, for at most `max` files. #[derive(Clone)] pub struct OnePerFile { pub max: usize, } #[async_trait] impl TransformResponse for OnePerFile { async fn transform_response( &self, mut query: Query, ) -> Result> { let mut seen_paths = HashSet::new(); let documents = query.documents_mut(); documents.retain(|document| { let path = document .metadata() .get("path") .and_then(|value| value.as_str()) .unwrap_or_default(); seen_paths.insert(path.to_string()) }); documents.truncate(self.max); Ok(query) } } ``` `TransformResponse` has one async method, `transform_response`. Implement it with `#[async_trait]`, from the `async-trait` crate or `swiftide::reexports::async_trait`. The trait requires `Clone` through `DynClone`, so derive it. This transformer edits the list in place through `documents_mut`. It keeps the first document per `path` metadata value, in the order the documents arrive, and caps the list at `max`. Documents without a `path` share one empty key, so only the first of them is kept. It does not call `transformed_response`, so `answers::Simple` still renders the documents one by one. Call `transformed_response` only when your transformer produces text for the answer to use in place of the documents, as `Summary` does. Place it after `then_retrieve`: ```rust let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_transform_response(OnePerFile { max: 8 }) .then_answer(answers::Simple::from_client(openai)); ``` For a small change, a closure works too. It must be synchronous, take a `Query` and return `anyhow::Result>`. It must also be `Clone + Send + Sync`, so capture only values that are themselves `Clone`: ```rust let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_transform_response(|mut query: Query| { query.documents_mut().truncate(5); Ok(query) }) .then_answer(answers::Simple::from_client(openai)); ``` ## Gotchas * In 0.33 the plain `Qdrant` retriever returns `Debug`-quoted content, so `Rerank` scores the quoted text. See the gotcha in [Retrieval and search strategies](/querying/retrieval-and-search-strategies/#gotchas) for the docs-qa workaround. * `Rerank` changes neither `current()` nor the transformation history. The `Retrieved` events in `history()` hold copies of the documents as the retriever returned them. Comparing them with `documents()` shows what was dropped only if no wrapper edited the documents after retrieval and the query retrieved once. * The `Ragas` evaluator records contexts before `Rerank` runs, so its dataset holds the retrieved candidates, not the kept documents; see [The dataset](/querying/evaluation/#the-dataset). A custom evaluator can read the reranked documents at its second call, after the answer. * To use another rerank model, build a `TextRerank` yourself, wrap it in `Arc::new(tokio::sync::Mutex::new(..))` and pass it to `model`. That needs direct `fastembed` and `tokio` dependencies, because Swiftide does not re-export `fastembed`. * Clones of a `Rerank` share one model behind a Tokio mutex. Scoring is synchronous CPU work, so concurrent queries, such as those from `query_all`, take turns at the rerank step. * If retrieval returns `top_k` documents or fewer, `Rerank` only reorders them. Retrieve more candidates than you keep. ## Reference * [`Rerank`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.Rerank.html) * [`RerankBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.RerankBuilder.html) * [`response_transformers::Summary`](https://docs.rs/swiftide/0.34/swiftide/query/response_transformers/struct.Summary.html) * [`TransformResponse`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.TransformResponse.html) * [`Query`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Query.html) * [`Document`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Document.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) * The [docs-qa showcase](/showcases/docs-qa/) and the [FastEmbed integration](/integrations/fastembed/) ## Next [Generating answers](/querying/answering/) # Retrieval and search strategies > Pick a search strategy (similarity, hybrid or custom), set top_k and filters, and pass it to a Qdrant, PgVector, LanceDB or DuckDB storage that acts as the retriever. A search strategy decides how many documents come back and how a retriever finds them. You pick a strategy, set `top_k` and a filter, and hand it to your storage, which acts as the retriever: Qdrant, PgVector, LanceDB or DuckDB. ## Similarity search examples/ask-your-codebase/src/query.rs ```rust let search = SimilaritySingleEmbedding::default().with_top_k(top_k); let pipeline = query::Pipeline::from_search_strategy(search) // Rephrase the question into several, to widen what the search can match .then_transform_query( GenerateSubquestions::builder() .client(llm.clone()) .prompt_template(SUBQUESTIONS_PROMPT.into()) .build()?, ) // Embed with the same local model that embedded the chunks .then_transform_query(Embed::from_client(embedding::local_model()?)) .then_retrieve(codebase.vector_store()?) // Answer the original question from the retrieved code, each chunk under its file path .then_answer( answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(CHUNK_TEMPLATE) .build()?, ); let answered = pipeline.query(question).await?; ``` `SimilaritySingleEmbedding` searches with the dense embedding that the `Embed` query transformer put on the query. It returns `top_k` documents. The default is 10. ask-your-codebase searches a LanceDB table and sets `top_k` from its `--top-k` flag, which defaults to 20. `with_top_k` takes the strategy by value and returns it, so it chains onto `default()` or `from_filter`. `SimilaritySingleEmbedding::default()` exists only for the unfiltered strategy, so its filter type is inferred as `()`, even on stores that also retrieve with a typed filter. For a filtered search, start from `from_filter` or call `with_filter`, as in [Filters](#filters). ## Hybrid search examples/docs-qa/src/query.rs ```rust query::Pipeline::from_search_strategy( HybridSearch::default() .with_top_n(CANDIDATES) .with_top_k(CANDIDATES), ) ``` `HybridSearch` combines a keyword or sparse search with a dense search. `top_n` caps the candidates of each search. `top_k` caps the documents that come back. Both default to 10. docs-qa fetches 20 candidates and later reranks them down to 10. `with_top_n`, `with_top_k`, `with_dense_vector_field` and `with_sparse_vector_field` take the strategy by value and return it, so they chain without `.to_owned()`. `HybridSearch` implements `Default` for every filter type. When nothing else fixes the filter type, name it, as docs-qa does with `HybridSearch` in its return type, or the compiler stops with E0283. On Qdrant, the query needs a dense and a sparse embedding, so the pipeline runs both query transformers before it retrieves. The example creates two FastEmbed models in its setup: `fastembed` for dense and `fastembed_sparse` for sparse embeddings. ```rust // Use sophisticated model for our query let openai = openai::OpenAI::builder() .default_prompt_model("gpt-4o") .build() .unwrap(); let query_pipeline = query::Pipeline::from_search_strategy( // Return a large amount of documents because we have a large context window // By default it uses the Combined fields, no need to configure HybridSearch::default().with_top_n(20).with_top_k(20), ) // Generate subquestions on the initial query to increase our query coverage .then_transform_query(query_transformers::GenerateSubquestions::from_client( openai.clone(), )) // Generate the same embeddings we used for indexing .then_transform_query(query_transformers::Embed::from_client(fastembed.clone())) .then_transform_query(query_transformers::SparseEmbed::from_client( fastembed_sparse.clone(), )) .then_retrieve(qdrant.clone()) // Answer with Simple, which either takes the documents as is (in this case), or any // transformations applied after querying .then_answer(answers::Simple::from_client(openai.clone())); ``` FastEmbed is the only sparse embedding model in Swiftide, so Qdrant hybrid search needs the `qdrant` and `fastembed` feature flags. The chunks need sparse embeddings too. Run the `SparseEmbed` transformer in the indexing pipeline next to `Embed`: ```rust indexing::Pipeline::from_loader(FileLoader::new("swiftide-core/").with_extensions(&["rs"])) // Chunk fairly large as the context window is big .then_chunk(ChunkCode::try_for_language_and_chunk_size( "rust", 10..2048, )?) // Generate metadata on the code chunks to increase our chances of finding the right code .then(MetadataQACode::from_client(openai.clone()).build().unwrap()) .then_in_batch( transformers::SparseEmbed::new(fastembed_sparse.clone()).with_batch_size(batch_size), ) .then_in_batch(transformers::Embed::new(fastembed.clone()).with_batch_size(batch_size)) .then_store_with(qdrant.clone()) .run() .await?; ``` Qdrant runs a sparse and a dense search, each limited to `top_n`. It merges them with reciprocal rank fusion (RRF) and keeps `top_k`. The collection needs a sparse vector next to the dense one; see [Qdrant](/integrations/qdrant/). DuckDB runs hybrid search without a sparse embedding. It scores the chunk text with BM25 full-text search over `query.current()` and the dense embedding with cosine similarity. Each score is normalized to its maximum, and the final score is 0.8 times the dense score plus 0.2 times the full-text score. After `GenerateSubquestions`, the keyword text is the whole list of generated questions. ## Filters A filter narrows the search to documents with matching metadata. Each storage takes its own filter type. Qdrant takes a `Filter` from its client, which Swiftide re-exports at `swiftide::integrations::qdrant::qdrant_client`: ```rust let only_readme = Filter::must([Condition::matches("path", "README.md".to_string())]); let pipeline = query::Pipeline::from_search_strategy(SimilaritySingleEmbedding::from_filter( only_readme.clone(), )) .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(openai)); // The same filter on a hybrid search applies to the dense and the sparse search let _hybrid = HybridSearch::from_filter(only_readme).with_top_n(20); ``` PgVector and LanceDB take a `String`. PgVector expects `key = value` and compares it with the metadata column `meta_`, so the key must be a metadata field on the table. Swiftide does not put the node path into metadata. To filter on `path`, insert it into each node’s metadata while you index (see [Transformers and metadata](/indexing/transformers-and-metadata/)), as ask-your-codebase does. Without it, PgVector refuses to store the node with “Missing metadata field”. ```rust // `path` must be a metadata field on the table: `PgVector::builder().with_metadata("path")`. // Swiftide does not add it for you; insert `path` into each node's metadata while indexing. let search = SimilaritySingleEmbedding::from_filter("path = README.md".to_string()).with_top_k(5); let pipeline = query::Pipeline::from_search_strategy(search) .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(pgvector) .then_answer(answers::Simple::from_client(openai)); ``` LanceDB passes the string to its `only_if` method, which takes a SQL predicate on the table’s columns, such as `path = 'README.md'`. That column has a value only if you put `path` in the node metadata and configured it with `with_metadata("path")`. DuckDB has no filter. ## Custom strategies `CustomStrategy` hands the query to your closure, and the closure builds the query the storage runs. `from_query` takes a synchronous closure, `from_async_query` an async one. PgVector runs a `sqlx::QueryBuilder`, LanceDB a LanceDB query, and DuckDB a SQL string. ```rust use sqlx::{Postgres, QueryBuilder}; // The default table is `swiftide_pgv_store`; `meta_path` exists only with `.with_metadata("path")` let table = pgvector.get_table_name().to_owned(); let keyword_search = CustomStrategy::from_query(move |query: &Query| { // The retriever reads `id`, `chunk` and every `meta_` column from each row let mut sql = QueryBuilder::::new(format!("SELECT id, chunk, meta_path FROM {table}")); sql.push(" WHERE to_tsvector('english', chunk) @@ plainto_tsquery('english', ") .push_bind(query.current().to_string()) .push(") LIMIT 10"); Ok(sql) }); let pipeline = query::Pipeline::from_search_strategy(keyword_search) .then_retrieve(pgvector) .then_answer(answers::Simple::from_client(openai)); ``` PgVector maps each row by column name. Select `id`, `chunk` and the `meta_` columns you need. `get_table_name()` returns the table, `swiftide_pgv_store` unless you set another. Swiftide does not re-export sqlx, so add it yourself: ```sh cargo add sqlx@0.9 --no-default-features --features postgres ``` A LanceDB custom strategy returns a query that implements `lancedb::query::ExecutableQuery`. Swiftide does not re-export lancedb either. Enable the `lancedb` feature flag and add the crate at the version Swiftide uses: ```sh cargo add lancedb@0.33 ``` DuckDB reads the first three columns of each row by position: the id, the chunk and the path. ## Retrievers per storage | Storage (feature flag) | Strategies | Filter | Limits | | ----------------------- | --------------------------------------------------------------------- | ----------------------- | -------------------------------------------------------------------------------------------------------------- | | `Qdrant` (`qdrant`) | `SimilaritySingleEmbedding`, `HybridSearch` | Qdrant `Filter` | Similarity search uses the combined vector when the collection has more than one vector or a sparse vector | | `PgVector` (`pgvector`) | `SimilaritySingleEmbedding`, `CustomStrategy>` | `String`, `key = value` | One vector field; orders by cosine distance | | `LanceDB` (`lancedb`) | `SimilaritySingleEmbedding`, `CustomStrategy` | `String`, SQL predicate | Exactly one vector field; content comes from the `chunk` column | | `Duckdb` (`duckdb`) | `SimilaritySingleEmbedding`, `HybridSearch`, `CustomStrategy` | none | Uses one vector, picked arbitrarily when several are configured; documents carry `id` and `path` metadata only | Setup for each storage is on [Qdrant](/integrations/qdrant/), [PgVector](/integrations/pgvector/) and [LanceDB](/integrations/lancedb/). DuckDB has no integration page; see the `Duckdb` reference below. ## Custom retrievers A retriever implements `Retrieve` for a search strategy `S`. It receives the strategy and the pending query, and returns `query.retrieved_documents(documents)`. `retrieved_documents` adds the documents and records a retrieval in the query’s transformation history. The trait requires `Clone`. ```rust /// Keeps the documents that share a word with the question. No store, no embeddings. #[derive(Clone)] struct KeywordRetriever { documents: Vec, } #[async_trait] impl Retrieve for KeywordRetriever { async fn retrieve( &self, search_strategy: &SimilaritySingleEmbedding, query: Query, ) -> anyhow::Result> { let words: Vec = query .current() .split_whitespace() .map(str::to_lowercase) .collect(); let matches = self .documents .iter() .filter(|document| { let content = document.content().to_lowercase(); words.iter().any(|word| content.contains(word.as_str())) }) .take(usize::try_from(search_strategy.top_k())?) .cloned() .collect(); Ok(query.retrieved_documents(matches)) } } ``` ```rust let retriever = KeywordRetriever { documents: vec![ Document::from("Swiftide indexes data with an indexing pipeline."), Document::from("A query pipeline answers questions over indexed data."), ], }; let pipeline = query::Pipeline::default() .then_retrieve(retriever) .then_answer(answers::Simple::from_client(openai)); ``` A closure works too: `then_retrieve` accepts any `Fn(&S, Query) -> Result>` that is `Send + Sync + Clone`. The closure is synchronous. ## Gotchas * Retrieval fails when the query has no embedding. Qdrant and LanceDB report “No embedding for query”, PgVector and DuckDB “Missing embedding in query state”. Qdrant hybrid search also needs a sparse embedding (“No sparse embedding for query”). * `HybridSearchBuilder::default()` sets `top_k` and `top_n` to 0. Start from `HybridSearch::default()`, which sets both to 10. * Qdrant only implements hybrid search for `HybridSearch`. Write that type in signatures, as docs-qa does, even without a filter. * Qdrant returns each document’s content `Debug`-quoted: wrapped in `"` with escaped newlines. docs-qa wraps the retriever in `CleanQdrant` ([Qdrant content workaround](/showcases/docs-qa/#qdrant-content-workaround)), which restores the text before reranking and answering. * DuckDB hybrid search does not rank before it cuts. Each search takes the first `top_n` rows of the table, unranked, and only rows in both lists get scored. On a table larger than `top_n`, most chunks are never considered. Set `top_n` to at least the row count, or use `SimilaritySingleEmbedding`. * A small `top_k` hides relevant chunks. In ask-your-codebase, one answer had to infer a detail because the chunk that held it was not among the 20 retrieved. Raise `top_k` for broad questions, or retrieve more and [rerank](/querying/reranking-and-response-transformers/). ## Reference * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`HybridSearch`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.HybridSearch.html), [`HybridSearchBuilder`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.HybridSearchBuilder.html) * [`CustomStrategy`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.CustomStrategy.html) * [`Retrieve`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Retrieve.html) * [`SearchStrategy`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SearchStrategy.html) * [`SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.SparseEmbed.html), [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html), [`PgVector`](https://docs.rs/swiftide/0.34/swiftide/integrations/pgvector/struct.PgVector.html), [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html), [`Duckdb`](https://docs.rs/swiftide/0.34/swiftide/integrations/duckdb/struct.Duckdb.html) * Showcases: [ask-your-codebase](/showcases/ask-your-codebase/), [docs-qa](/showcases/docs-qa/) ## Next [Reranking and response transformers](/querying/reranking-and-response-transformers/) # Transforming queries > Rewrite a question into subquestions with GenerateSubquestions, embed it densely and sparsely with the model your chunks were embedded with, or write your own TransformQuery. A query pipeline can rewrite the user’s question before the retriever runs: a large language model (LLM) widens it into related questions, and an embedding model turns the result into the vectors your chunks were stored with. The query transformers need no feature flag. The snippets use the OpenAI client and Qdrant, which sit behind the `openai` and `qdrant` feature flags: ```sh cargo add swiftide@0.34 --features openai,qdrant cargo add async-trait anyhow ``` The second line is for the custom query transformer further down, which uses the `#[async_trait]` attribute and `anyhow::Result`. ## Subquestions ```rust let pipeline = query::Pipeline::default() .then_transform_query( GenerateSubquestions::builder() .client(openai.clone()) .num_questions(3) .build()?, ) // Embed the text that the step above produced, with the model the chunks were embedded with .then_transform_query(Embed::from_client(embedder)) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(openai)); ``` A query transformer runs before retrieval. It takes the pending query and returns a new pending query. You add one with `then_transform_query`, as often as you need, and they run in the order you add them. `GenerateSubquestions` sends the current query to an LLM through `SimplePrompt`. It asks for `num_questions` related questions, 5 by default. The reply replaces the current query, so the next step sees one text: the original question and its subquestions as a list. `GenerateSubquestions::from_client` builds it with the default prompt and count. The default prompt is written for code. It asks the model to think about the dependencies, usage and structure of the code. Replace it when you search other content. ### Subquestion prompts The ask-your-codebase showcase sets its own prompt on the builder: examples/ask-your-codebase/src/query.rs ```rust let search = SimilaritySingleEmbedding::default().with_top_k(top_k); let pipeline = query::Pipeline::from_search_strategy(search) // Rephrase the question into several, to widen what the search can match .then_transform_query( GenerateSubquestions::builder() .client(llm.clone()) .prompt_template(SUBQUESTIONS_PROMPT.into()) .build()?, ) // Embed with the same local model that embedded the chunks .then_transform_query(Embed::from_client(embedding::local_model()?)) .then_retrieve(codebase.vector_store()?) // Answer the original question from the retrieved code, each chunk under its file path .then_answer( answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(CHUNK_TEMPLATE) .build()?, ); let answered = pipeline.query(question).await?; ``` `SUBQUESTIONS_PROMPT` is a constant next to the pipeline: examples/ask-your-codebase/src/query.rs ````rust /// Widens the question into the follow-up questions its answer involves. /// /// They are phrased like the questions generated for each chunk while indexing, so the two /// embed close together. const SUBQUESTIONS_PROMPT: &str = r" Your job is to help a search find the code that answers a developer's question about a codebase. Question: {{ question }} Write {{ num_questions }} short follow-up questions that the code answering this question would also answer. Think about the mechanisms involved: the main loop, the types and enum variants, the conditions that are checked, the functions that are called, and the configuration that changes the outcome. ## Constraints * Phrase them the way a developer would ask about this codebase, e.g. 'When does the agent stop?'. * Do not guess names of types or functions. * Respond with the original question and the follow-up questions only, one per line, prefixed with '- '. "; /// Renders a retrieved chunk under its file path, so the answer can cite it. /// /// `metadata.path` is the `PATH_FIELD` metadata; a test keeps the two in step. const CHUNK_TEMPLATE: &str = r"### {{ metadata.path }} ``` {{ content }} ``` "; /// Answers from the retrieved code and cites files. const ANSWER_PROMPT: &str = r" Answer a developer's question about a codebase, using snippets of its source code. Question: {{ question }} ## Constraints * Only use the snippets below. If they do not answer the question, say briefly what is missing. * Be concrete: name the types, functions and variants involved and explain when they apply. * Describe behaviour: what calls what, in which order, and under which conditions. * Cite the file each statement comes from in backticks, e.g. (`src/lib.rs`). * Use concise markdown. Add a short code example only if it shows how to use the API. ## Snippets {{ documents }} "; ```` The field takes a `Prompt`, so convert a `&'static str` or `String` with `.into()` or `Prompt::from`. A borrowed `&str` built at runtime needs `.to_string()` first. The template receives two values: `question`, the current query, and `num_questions`. The prompt phrases the subquestions like the questions that its indexing pipeline wrote for each chunk. Similar phrasing puts the query vector close to the chunk vectors. Keep the instruction to repeat the original question: the next step embeds only what the model returns. [Prompts and templates](/models/prompts-and-templates/) covers the template syntax. ## Dense and sparse embeddings examples/docs-qa/src/query.rs ```rust pipeline // Hybrid search needs both a dense and a sparse embedding of the question .then_transform_query(query_transformers::Embed::from_client( self.embedders.dense.clone(), )) .then_transform_query(query_transformers::SparseEmbed::from_client( self.embedders.sparse.clone(), )) .then_retrieve(self.qdrant.clone()) // A local cross-encoder reorders the fused candidates; only the best few remain .then_transform_response(self.reranker.clone()) .then_answer(self.answerer.clone()) ``` `query_transformers::Embed` embeds the current query with an `EmbeddingModel` and stores the vector in `query.embedding`. `query_transformers::SparseEmbed` does the same with a `SparseEmbeddingModel` and fills `query.sparse_embedding`. At this release only `FastEmbed`, behind the `fastembed` feature flag, implements `SparseEmbeddingModel`. Use the models that your [indexing pipeline embedded the chunks with](/indexing/embedding/). Vectors from different models do not compare. The docs-qa showcase creates its dense and sparse `FastEmbed` models together, and loads the same two models for its indexing and its query pipeline. Similarity search needs `embedding`. Hybrid search on Qdrant needs both, and fails with `No sparse embedding for query` when the sparse one is missing. [Retrieval and search strategies](/querying/retrieval-and-search-strategies/) covers both strategies. An embedding step reads the current query at the moment it runs. Put it after every step that rewrites the text. If `Embed` runs before `GenerateSubquestions`, the vector search never sees the subquestions. ## Custom query transformers ```rust /// Puts the instruction that BGE embedding models expect for search queries in front of the /// question. #[derive(Clone)] pub struct BgeQueryInstruction; #[async_trait] impl TransformQuery for BgeQueryInstruction { async fn transform_query( &self, mut query: Query, ) -> anyhow::Result> { let instructed = format!( "Represent this sentence for searching relevant passages: {}", query.current() ); query.transformed_query(instructed); Ok(query) } } ``` `TransformQuery` has one async method, `transform_query`. It takes the `Query` by value and returns it. Implement it with the `#[async_trait]` attribute from the `async-trait` crate. The trait requires `Send`, `Sync` and `DynClone`, so derive `Clone`. The current query has no setter of its own. `transformed_query` replaces it and records a `Transformed` event with the text before and after. The original question never changes. The fields `embedding`, `sparse_embedding` and `documents` are public, so a transformer that computes a vector sets it directly, as `Embed` does. This example prepares a question for an embedding model of the BGE (BAAI General Embedding) family, which suggests an instruction in front of short search queries. The chunks are embedded without it. ```rust let pipeline = query::Pipeline::default() // A closure is enough for a step without state .then_transform_query(|mut query: Query| { let trimmed = query.current().trim().to_string(); query.transformed_query(trimmed); Ok(query) }) .then_transform_query(BgeQueryInstruction) .then_transform_query(Embed::from_client(embedder)) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(openai)); ``` A closure is a query transformer too. It must be `Fn(Query) -> Result>` and `Clone`, `Send` and `Sync`. Closures are synchronous, so use a struct for a step that awaits. The pipeline logs `Transformed query` at debug level after each step, with the transformer’s `name()` and the new current query. `name()` defaults to the last path segment of the type name, so a closure logs a generic closure name instead of a useful one. Override `name()` on a struct to get a clearer log line. ## Transformed query after retrieval ```rust let answered = pipeline.query("How does an agent decide to stop?").await?; for event in answered.history() { if let Some(documents) = event.documents() { println!( "retrieved {} documents for:\n{}", documents.len(), event.before() ); } else { println!("rewrote:\n{}\ninto:\n{}", event.before(), event.after()); } } ``` Since 0.33 the current query keeps the transformed text after retrieval. The `Retrieved` event records the current query at the moment of retrieval, as both `before()` and `after()`, and the documents it returned. Before 0.33, retrieval cleared the current query. That text is not always what the search used. A vector retriever such as Qdrant searches with `query.embedding`, which `Embed` computed when it ran. A step after `Embed` that rewrites the text changes the event, not the vector. `history()` lists the events in order. Each `transformed_query` call adds a `Transformed` event, and each retrieval adds a `Retrieved` event. `Embed` and `SparseEmbed` add nothing, because they leave the text alone. Once the query is answered, `current()` returns the answer. Read the transformed query from the history instead. [Generating answers](/querying/answering/) shows how the answer generator uses the original and the transformed query. ## Gotchas * Two types are called `Embed`. `query_transformers::Embed` belongs in a query pipeline, `indexing::transformers::Embed` in an indexing pipeline. The same goes for `SparseEmbed`. Import them by module path. * After `GenerateSubquestions`, `Embed` turns the whole list into one vector. The pipeline does not run one search per subquestion. * DuckDB hybrid search, behind the `duckdb` feature flag, uses the subquestion list as its keyword text; see [Retrieval and search strategies](/querying/retrieval-and-search-strategies/). * The FastEmbed [`Rerank` response transformer](/querying/reranking-and-response-transformers/) scores documents against `original()`, and the [`Ragas` evaluator](/querying/evaluation/) matches rows by `original()`. Rewriting the question changes neither. * `Query::new` sets only the original question and leaves the current query empty, so an embedding step would embed an empty string. Pass a `&str` or `String` to `query`, or build the query with `Query::from`, which sets both. ## Reference * [`Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) (`then_transform_query`) * [`TransformQuery`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.TransformQuery.html) * [`GenerateSubquestions`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.GenerateSubquestions.html) * [`query_transformers::Embed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.Embed.html) * [`query_transformers::SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.SparseEmbed.html) * [`Query`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Query.html) * [`TransformationEvent`](https://docs.rs/swiftide/0.34/swiftide/query/enum.TransformationEvent.html) * [`states::Pending`](https://docs.rs/swiftide/0.34/swiftide/query/states/struct.Pending.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`SparseEmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SparseEmbeddingModel.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) * Showcases: [ask-your-codebase](/showcases/ask-your-codebase/) and [docs-qa](/showcases/docs-qa/) ## Next [Retrieval and search strategies](/querying/retrieval-and-search-strategies/) # Agents as task nodes > Run an agent as a task node with TaskAgent, or implement TaskNode to turn its stop payload into typed output. You wrap an agent as a task node, first with the `TaskAgent` adapter and then as your own `TaskNode` that hands the next node a typed value read from the agent’s stop payload. `TaskAgent` is the shortest form: ```rust /// Adds an entry to the top of `CHANGELOG.md`. /// /// # Arguments /// - `entry`: The Markdown entry to add, including its heading. #[swiftide::tool] async fn add_changelog_entry( context: &dyn AgentContext, entry: &str, ) -> Result { let executor = context.executor(); let changelog = executor .exec_cmd(&Command::read_file("CHANGELOG.md")) .await? .into_string_lossy(); executor .exec_cmd(&Command::write_file( "CHANGELOG.md", format!("{entry}\n\n{changelog}"), )) .await?; Ok(ToolOutput::text("Entry added.")) } /// An agent wrapped in `TaskAgent` as the only node of a task. pub async fn update_changelog(openai: OpenAI) -> anyhow::Result<()> { let checkout: Arc = Arc::new(LocalExecutor::new("/srv/checkout")); let agent = Agent::builder() .llm(&openai) .context(DefaultContext::from_executor(checkout)) .system_prompt("You keep the changelog of this repository up to date.") .tools([add_changelog_entry()]) .build()?; let mut task = Task::::new(); let changelog = task.register_node(TaskAgent::from(agent)); task.starts_with(changelog); task.register_transition(changelog, task.transitions_to_finish())?; // `TaskAgent` returns `()`: the result is the file the tool wrote. match task.run("Add an entry for the 0.33 release.").await? { TaskRunOutcome::Completed(()) => println!("CHANGELOG.md is updated"), TaskRunOutcome::Paused => unreachable!("no node in this task pauses"), } Ok(()) } ``` `TaskAgent::from(agent)` turns the agent into a task node. Its input is a `Prompt`, its output is `()` and its error is `AgentError`. Each evaluation calls `query` on the agent with the prompt. `TaskAgent` gives you no way to read the agent’s answer. It fits agents whose work is a side effect, like the changelog entry that `add_changelog_entry` writes through the tool executor. When the next node needs data from the agent, write your own node. The adapter keeps the agent behind an `Arc>`. Branches that share one `TaskAgent` run one at a time and add to the same [message history](/agents/context-and-history/#message-history). The `swiftide-agents` feature flag turns on `swiftide-tasks` as well: ```sh cargo add swiftide@0.34 --features swiftide-agents,openai ``` The custom node below also needs serde and schemars 1 (the version swiftide re-exports as `swiftide::reexports::schemars`), plus `thiserror` for its error type: ```sh cargo add serde --features derive cargo add schemars@1 --features derive cargo add serde_json thiserror ``` ## Custom agent node The workspace-architect showcase fans out one agent per publishable crate of a Cargo workspace, and each branch returns a `CrateSummary`. The node holds the model client, not an agent: examples/workspace-architect/src/analyst.rs ```rust /// A task node that documents one workspace crate with a freshly built agent. /// /// The node is registered once and used by every fan-out branch: each branch evaluates it with a /// different [`WorkspaceCrate`], and every evaluation builds its own agent scoped to that crate's /// directory, so branches share nothing and can run in parallel. #[derive(Clone)] pub struct CrateAnalyst { llm: L, progress: Progress, } impl CrateAnalyst { pub fn new(llm: L, progress: Progress) -> Self { Self { llm, progress } } } ``` The `TaskNode` implementation builds an agent, queries it and reads what the agent passed to `stop`: examples/workspace-architect/src/analyst.rs ```rust /// How often an agent is told to fix a missing or malformed report before the crate fails. const MAX_CORRECTIONS: usize = 2; #[async_trait] impl TaskNode for CrateAnalyst { type Input = WorkspaceCrate; type Output = CrateSummary; type Error = AnalystError; async fn evaluate( &self, _node_id: &DynNodeId, member: &WorkspaceCrate, ) -> Result { self.analyse(member).await.map_err(|cause| AnalystError { crate_name: member.name.clone(), cause, }) } } impl CrateAnalyst { async fn analyse(&self, member: &WorkspaceCrate) -> Result { let analysis = self.progress.analysis(&member.name); let mut agent = self.agent_for(member, analysis.clone())?; agent.query(brief(member)).await?; // Models occasionally answer in prose or nest the `stop` arguments; say what went wrong. let mut corrections = 0; let report = loop { match submitted_report(&agent) { Ok(report) => break report, Err(problem) if corrections < MAX_CORRECTIONS => { corrections += 1; analysis.corrected(&problem); agent.query(problem.correction()).await?; } Err(problem) => return Err(problem.into()), } }; analysis.finished(&report); Ok(report.reconciled_with(member)) } } /// The report the agent passed to `stop`, checked against the `CrateReport` schema. fn submitted_report(agent: &Agent) -> Result { match agent.stop_reason() { Some(StopReason::RequestedByTool(_, Some(arguments))) => { Ok(CrateReport::deserialize(arguments)?) } other => Err(ReportProblem::Missing { stop_reason: other.cloned(), }), } } ``` `evaluate` borrows its input and reaches the node through `&self`. Every branch of the fan-out evaluates the same node instance. That is why the agent is a local variable of `analyse` and not a field. After `query` returns, `agent.stop_reason()` says why the agent stopped. A call to the stop tool with arguments gives `StopReason::RequestedByTool(_, Some(arguments))`. `CrateReport::deserialize` reads the report straight from the borrowed `serde_json::Value`. Any other stop reason, or a payload that does not match, becomes a `ReportProblem`. The node then checks the report against `cargo metadata` and returns a `CrateSummary`. The transitions that follow are typed, so the compiler checks that the next node accepts it. ## Stop schema The agent’s stop tool requires a `CrateReport`: examples/workspace-architect/src/summary.rs ```rust /// What an analyst agent reports about one crate when it calls `stop`. /// /// The doc comments become field descriptions in the JSON schema the model sees, so they double /// as instructions. #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] pub struct CrateReport { /// Two to four sentences: the problem this crate solves and the role it plays in the /// workspace. Describe responsibilities, not files. pub purpose: String, /// Four to eight of the most important public items a newcomer must know (traits, structs, /// enums, macros or functions), each formatted as "`Name`: what it is for". Use names exactly /// as they appear in the source. pub key_types: Vec, /// One entry per workspace crate listed as a dependency in the brief. Empty if the brief /// lists none. pub depends_on: Vec, /// One to three sentences on feature flags, extension points (traits meant to be /// implemented) and non-obvious design decisions. pub notes: String, } /// How the documented crate uses one of its workspace dependencies. #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] pub struct DependencyUsage { /// The dependency's crate name, exactly as listed in the brief. pub crate_name: String, /// What the documented crate uses it for. pub usage: String, } /// A `stop` tool whose arguments must match [`CrateReport`]. pub fn summary_stop_tool() -> StopWithArgs { StopWithArgs::with_parameters_schema(schema_for!(CrateReport)) } ``` `summary_stop_tool` replaces the default `stop` tool with one whose arguments must match `CrateReport`. [Stopping and structured results](/agents/stopping-and-structured-results/) explains how the payload reaches `StopReason::RequestedByTool`. ## One agent per input `agent_for` builds a fresh agent for every evaluation: examples/workspace-architect/src/analyst.rs ```rust fn agent_for( &self, member: &WorkspaceCrate, analysis: Analysis, ) -> Result { // Every command the tools run executes inside the crate directory. let executor: Arc = Arc::new(LocalExecutor::new(&member.dir)); let tools: [Box; 4] = [ list_files(), read_file(), grep(), summary_stop_tool().into(), ]; Agent::builder() .name(member.name.as_str()) .llm(&self.llm) .context(DefaultContext::from_executor(executor)) .system_prompt(SYSTEM_PROMPT) // Replace the default `stop` tool with one that requires a `CrateReport`. .without_default_stop_tool() .tools(tools) .before_tool(move |_agent, tool_call| { analysis.tool_called(tool_call.name(), tool_call.args().unwrap_or_default()); Box::pin(async { Ok(()) }) }) .limit(MAX_TURNS) .build() } ``` Each agent gets a [`LocalExecutor`](/agents/executors/#local-executor) whose working directory is its crate’s directory, so relative paths in its tools resolve inside that crate. This is not a sandbox. Absolute paths and `..` still reach the rest of the file system. The showcase’s own tools reject absolute paths and `..`, but they follow symlinks. `without_default_stop_tool()` takes effect in the next `.tools(...)` call, which builds the tool set without the default `stop`. `summary_stop_tool()` then adds the one bound to `CrateReport`. The order matters, as the gotchas below explain. The `before_tool` [hook](/agents/hooks/) reports progress. `limit(MAX_TURNS)` caps the completions of a single query at 30. The branches share no agent, executor or message history. That makes it safe to run them under `ConcurrencyModel::Parallel`, as [Fan-out and join](/tasks/fan-out-and-join/#concurrency-model) shows. ## Corrections A model sometimes answers in prose or nests its report. The node then queries the same agent again with a correction, up to `MAX_CORRECTIONS` (2) times. For a payload that does not match the schema, the correction quotes the serde error. For a missing stop, it is a fixed reminder to call `stop` now. The agent keeps its message history between the queries, so it sees its own mistake. [Workspace architect](/showcases/workspace-architect/#output) shows the captured retries. ## Shared agent `examples/tasks.rs` keeps one agent for the life of the node instead: ```rust /// A domain-specific task step around an `Agent`. /// /// We do not use `TaskAgent` here on purpose. `TaskAgent` is useful when you simply want /// "run this prompt through an agent" behavior. In many applications, however, an agent step has /// its own responsibility and its own input/output shape. That is when implementing `TaskNode` /// directly becomes the clearer API. #[derive(Clone, Debug)] struct BriefingAgent { agent: Arc>, } impl BriefingAgent { fn new(agent: Agent) -> Self { Self { agent: Arc::new(Mutex::new(agent)), } } } /// The structured payload that our custom stop tool expects from the agent. /// /// Returning a typed value like this is the main reason to implement a task node around an agent: /// downstream task steps can now work with domain data instead of unstructured free text. #[derive(Clone, Debug, Deserialize, Serialize, JsonSchema)] #[schemars(deny_unknown_fields)] struct BriefingDecision { audience: String, summary: String, next_steps: Vec, } #[derive(Debug)] enum BriefingAgentError { Agent(AgentError), MissingStructuredStop, InvalidStructuredStop(serde_json::Error), } impl std::fmt::Display for BriefingAgentError { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { match self { Self::Agent(error) => write!(f, "{error}"), Self::MissingStructuredStop => { write!(f, "agent stopped without the structured stop payload") } Self::InvalidStructuredStop(error) => { write!( f, "agent returned an invalid structured stop payload: {error}" ) } } } } impl std::error::Error for BriefingAgentError {} impl From for BriefingAgentError { fn from(value: AgentError) -> Self { Self::Agent(value) } } impl From for BriefingAgentError { fn from(value: serde_json::Error) -> Self { Self::InvalidStructuredStop(value) } } #[swiftide::reexports::async_trait::async_trait] impl TaskNode for BriefingAgent { type Input = String; type Output = BriefingDecision; type Error = BriefingAgentError; async fn evaluate( &self, _node_id: &NodeId< dyn TaskNode, >, brief: &Self::Input, ) -> Result { // The previous node produced a short brief. This node asks the agent to turn that brief // into a structured hand-off plan and to stop with a JSON payload we can deserialize. let handoff_prompt = Prompt::from( "Turn this brief into a structured hand-off plan.\n\ Brief:\n\ {{brief}}\n\n\ Call the `stop` tool once you have produced the final plan.", ) .with_context_value("brief", brief.clone()); let mut agent = self.agent.lock().await; agent.query_once(handoff_prompt).await?; let Some(StopReason::RequestedByTool(_, Some(payload))) = agent.stop_reason() else { return Err(BriefingAgentError::MissingStructuredStop); }; Ok(serde_json::from_value(payload.clone())?) } } ``` `BriefingAgent` locks the agent, calls `query_once` and deserializes the stop payload into a `BriefingDecision`. `query_once` runs one turn of the agent loop instead of looping until the agent stops. If the model calls another tool in that turn, the agent stops with `StopReason::NoNewMessages` and the node returns `MissingStructuredStop`. Use `query` for an agent that needs tools before it stops. The mutex serializes evaluations, and the message history carries over from one evaluation to the next. Use this shape when the node runs once per task, or when later evaluations should see earlier ones. ## Gotchas * Every call to `query` starts the `limit` count from zero. With corrections, one crate in the showcase can use up to `(1 + MAX_CORRECTIONS) * MAX_TURNS` completions. * An agent that reaches its limit stops with `StopReason::NoNewMessages`, not with a stop payload. The payload itself is an `Option`, so handle `None` too. * Call `without_default_stop_tool()` before `.tools(...)`. Called after it, or without any `.tools(...)` call, the default `stop` stays. Tools are unique by name, so your own `stop` passed later is dropped and the agent stops with `RequestedByTool(_, None)`. * `build()` returns an `AgentBuilderError`, and `query` returns an `AgentError`. Wrap both in one error type that names the input. `AnalystError` prints “analysing `{crate_name}` failed: …”, so a failed branch says which crate it was working on. ## Reference * [`TaskAgent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.TaskAgent.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html), [`Agent::query_once`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html#method.query_once) and [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`StopWithArgs`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.StopWithArgs.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) and [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html) and [`TaskRunOutcome`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskRunOutcome.html) * [`TaskNode`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.TaskNode.html) and [`DynNodeId`](https://docs.rs/swiftide/0.34/swiftide/tasks/type.DynNodeId.html) * [`ConcurrencyModel`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.ConcurrencyModel.html) ## Next [Built-in nodes](/tasks/built-in-nodes/) # Built-in nodes > Register a SimplePrompt, ChatCompletion or ToolExecutor trait object, or a plain closure, as a task node without writing a type. You end with a model call, a shell command and a reusable closure each running as a task node, and a rule for when to write your own `TaskNode` instead. Swiftide implements `TaskNode` for `Arc` and `Box` trait objects of `SimplePrompt`, `ChatCompletion` and `ToolExecutor`, and for the `SyncFn` and `AsyncFn` closure wrappers. The snippets use the OpenAI client and the local tool executor, so enable those feature flags next to `swiftide-tasks`: ```sh cargo add swiftide@0.34 --features swiftide-tasks,swiftide-agents,openai ``` ## Prompt nodes A closure builds a `Prompt`, and the model client answers it: ```rust let mut task: Task = Task::new(); let ask = task.register_node_fn(|topic: &String| { Ok::<_, NodeError>( Prompt::from("Explain {{ topic }} in two sentences.") .with_context_value("topic", topic.as_str()), ) }); // The cast turns the concrete client into a node: `Prompt` in, `String` out. let answer = task.register_node(Arc::new(openai) as Arc); task.starts_with(ask); task.register_transition(ask, move |prompt| answer.transitions_with(prompt))?; task.register_transition(answer, task.transitions_to_finish())?; if let TaskRunOutcome::Completed(text) = task.run("a task node").await? { println!("{text}"); } ``` The closure fills a template and returns the `Prompt`. The next node is the model client itself. The cast matters. Swiftide implements `TaskNode` for `Arc` and `Box`, not for each client type. `register_node(openai)` on its own does not compile, because the concrete client is not a node. A `SimplePrompt` node has `Input = Prompt`, `Output = String` and `Error = LanguageModelError`. Each evaluation clones the input `Prompt` and calls `prompt` on the client, which renders the template. See [Prompts and templates](/models/prompts-and-templates/) for building the `Prompt`. One `Arc` can back several nodes, as in the `examples/tasks.rs` excerpt in [Nodes and transitions](/tasks/nodes-and-transitions/#transitions). The workspace-architect showcase uses the same pattern. Its join node builds one `Prompt` from every crate summary, and a `SimplePrompt` node renders it into an `ARCHITECTURE.md` draft. In the approved run from `OUTPUT.md`, the render step took 28 seconds for an 11 KB draft. The model was `claude-sonnet-5-5`, called through Chat Completions on an OpenAI-compatible gateway (Bifrost in front of AWS Bedrock): ```text [ 40.5s] join 10 summaries; 230s of agent work in 41s wall-clock (5.7x) [ 40.5s] render drafting ARCHITECTURE.md [ 68.8s] review draft ready (11210 bytes), waiting for approval ``` ## Chat completion nodes When a step needs a system message, a full message history or tool specs, use a `ChatCompletion` node instead: ```rust let mut task: Task = Task::new(); let ask = task.register_node_fn(|question: &String| { Ok::<_, NodeError>(ChatCompletionRequest::from(vec![ ChatMessage::new_system("Answer in one sentence."), ChatMessage::new_user(question.clone()), ])) }); let chat: Arc = Arc::new(openai); let complete = task.register_node(chat); let answer = task.register_node_fn(|response: &ChatCompletionResponse| { Ok::<_, NodeError>(response.message().unwrap_or_default().to_string()) }); task.starts_with(ask); task.register_transition(ask, move |request| complete.transitions_with(request))?; task.register_transition(complete, move |response| answer.transitions_with(response))?; task.register_transition(answer, task.transitions_to_finish())?; if let TaskRunOutcome::Completed(text) = task.run("What does a task node do?").await? { println!("{text}"); } ``` An `Arc` or `Box` node has `Input = ChatCompletionRequest<'static>`, `Output = ChatCompletionResponse` and `Error = LanguageModelError`. It calls `complete` with the request. Task payloads must be `'static`, so the request has to own its messages. `ChatCompletionRequest::from(Vec)` does that, and so does `ChatCompletionRequest::builder()` with an owned `Vec`. To keep a borrowed request, call `to_owned()` on it first. The response is the whole `ChatCompletionResponse`. A node after it reads `message()`, which returns `Option<&str>`, or `tool_calls()`. The node does not run tools. To act on tool calls in a loop, use an agent. [Models overview](/models/overview/) covers the clients that implement `ChatCompletion`. ## Tool executor nodes A `ToolExecutor` node runs one command. Pair it with a closure that builds the `Command`: ```rust let mut task: Task = Task::new(); let command = task.register_node_fn(|count: &usize| { Ok::<_, NodeError>(Command::shell(format!("git log --oneline -n {count}"))) }); let shell: Arc = Arc::new(LocalExecutor::new(".")); let run = task.register_node(shell); task.starts_with(command); task.register_transition(command, move |cmd| run.transitions_with(cmd))?; task.register_transition(run, task.transitions_to_finish())?; match task.run(5_usize).await { Ok(TaskRunOutcome::Completed(output)) => { for chunk in output.stdout() { print!("{}", String::from_utf8_lossy(chunk)); } } Ok(TaskRunOutcome::Paused) => unreachable!("no node in this task pauses"), Err(TaskError::NodeError(failed)) => { // An exit status, a spawn failure or an I/O error. The text can sit in either stream. if let Some(CommandError::NonZeroExit(output)) = failed.node_error.downcast_ref() { eprintln!("{}", output.to_string_lossy()); } return Err(TaskError::NodeError(failed).into()); } Err(other) => return Err(other.into()), } ``` An `Arc` or `Box` node has `Input = Command`, `Output = CommandOutput` and `Error = CommandError`. It calls `exec_cmd` on the executor. `LocalExecutor` runs commands on the local machine. [Executors](/agents/executors/) covers the executors and their commands. Build the input with `Command::shell`, `Command::read_file` or `Command::write_file`. Chain `with_current_dir` or `with_timeout` to set a directory or a time limit for one command. Since 0.33, `CommandOutput` keeps standard output (stdout) and standard error (stderr) apart. `stdout()` and `stderr()` iterate over the chunks of one stream, in the order the executor read them. `to_string_lossy()` and `into_string_lossy()` merge both streams into one text. A command that exits with a non-zero status does not produce output for the next node. `LocalExecutor` returns `CommandError::NonZeroExit` with the captured output, and the run fails with `TaskError::NodeError`. Its `node_error` field holds the original `CommandError`, so `downcast_ref` gets the output back, as the snippet does. `NonZeroExit` also carries every I/O error, such as a command that fails to start or a missing file for `Command::read_file`. In that case the error text sits in a stdout chunk, not in stderr. The snippet prints both streams with `to_string_lossy()` for that reason. A command that runs past its timeout fails with `CommandError::TimedOut`, which the snippet passes on with the rest. ## Closure adapters `register_node_fn` wraps a closure in `SyncFn`, and `register_node_async_fn` wraps one in `AsyncFn`. Both types are public. Build one yourself when you want to name the node’s type, store it in a struct, or register the same adapter more than once: ```rust let mut task: Task = Task::new(); let double = SyncFn::new(|n: &i32| Ok::<_, NodeError>(n * 2)); let first = task.register_node(double.clone()); let second = task.register_node(double); task.starts_with(first); task.register_transition(first, move |n| second.transitions_with(n))?; task.register_transition(second, task.transitions_to_finish())?; assert_eq!(task.run(3).await?, TaskRunOutcome::Completed(12)); ``` `SyncFn::new` takes a closure `Fn(&I) -> Result`. `AsyncFn::new` takes a closure that returns `Box::pin(async move { .. })`. Both are `Clone`, and every `register_node` call adds a separate node. The error type defaults to `NodeError`, but a closure still needs its error type written out; see [Closure nodes](/tasks/nodes-and-transitions/#closure-nodes). ## Custom node or built-in Use a built-in node when the input and output types already fit the step. A prompt in and text out is a `SimplePrompt` node. A command in and its output out is a `ToolExecutor` node. Small conversions between steps are closures. Implement `TaskNode` yourself when the step: * takes or returns your own domain types * calls more than one service, or calls one in a loop * keeps state that the caller reads from outside the task * needs structured output from a model * should fail with an error that names its input The showcase’s `CrateAnalyst` runs an agent and returns a typed `CrateSummary`. Its `ApprovalGate` keeps a draft that the command-line interface (CLI) approves from outside. Both are custom nodes. [Agents as task nodes](/tasks/agents-as-nodes/#custom-agent-node) and [Pause and resume](/tasks/pause-and-resume/#approval-gate) show them. ## Gotchas * A `SimplePrompt` node clones its `Prompt` on every evaluation. Keep large values out of the template context if the node runs often. * A `ToolExecutor` node runs commands in the executor’s working directory. With `LocalExecutor`, a relative `with_current_dir` path resolves against that directory. * To branch on a failed command instead of failing the run, write your own node that calls the executor and returns an enum. Node outputs must be `Clone`, and `CommandError` is not, so map the error to a type of your own first. ## Reference * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`ChatCompletionRequest`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionRequest.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`LanguageModelError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.LanguageModelError.html) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [`Command`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.Command.html) * [`CommandOutput`](https://docs.rs/swiftide/0.34/swiftide/traits/struct.CommandOutput.html) * [`CommandError`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.CommandError.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) * [`SyncFn`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.SyncFn.html) * [`AsyncFn`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.AsyncFn.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`TaskNode`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.TaskNode.html) * [`NodeError`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeError.html) ## Next [How indexing works](/indexing/overview/) # Fan-out and join > Fan a task out into one branch per item, optionally run the branches in parallel, and collect their typed outputs in one join node. You end up with a task that runs one task node per item of a list, all at the same time, and collects every result in a single join node. The task types need the `swiftide-tasks` feature flag ([Tasks overview](/tasks/overview/#feature-flag)). The code comes from the workspace-architect showcase, which analyses each crate of a Cargo workspace with its own agent. examples/workspace-architect/src/graph.rs ```rust // One branch per crate, all evaluating the same analyst node with a different input. task.register_transition(discover, move |members: WorkspaceMembers| { let mut crates = members.crates.into_iter(); let Some(first) = crates.next() else { return Transition::error("the workspace has no crates to document"); }; crates .fold(Transition::fan_out(&analyse, first), |fan_out, member| { fan_out.and(&analyse, member) }) .concurrency_model(ConcurrencyModel::Parallel) .join_with(join.join()) })?; ``` examples/workspace-architect/src/graph.rs ```rust // Each branch hands its `CrateSummary` to the join; the join runs once all have arrived. task.register_transition(analyse, join.join())?; task.register_transition(join, { let progress = progress.clone(); move |prompt| { progress.step("render", "drafting ARCHITECTURE.md"); render.transitions_with(prompt) } })?; ``` The `discover` node returns the workspace members, and its transition folds them into one fan-out with a branch per crate. Every branch evaluates the same `analyse` node with a different `WorkspaceCrate`, in parallel, and `join_with` names the node that waits for the whole group. The `analyse` node registers one transition to the join target, `task.register_transition(analyse, join.join())`, and every branch uses it. Transitions belong to nodes, not branches, and each node has [exactly one](/tasks/nodes-and-transitions/#transitions). The join node is a closure over `&JoinInput`, registered with `task.register_node_fn` in the `nodes` region of `graph.rs`. It runs once every branch has delivered its `CrateSummary`, and its `Prompt` goes to the render node with an ordinary `transitions_with`. [Built-in nodes](/tasks/built-in-nodes/#prompt-nodes) covers the render node. ## Fan-out `Transition::fan_out(&node, input)` starts a `FanOutTransition` with one branch. `.and(&node, input)` adds another. Branches can target the same node or different nodes. Each input has to convert into the target node’s input type, or the code does not compile. A `FanOutTransition` becomes a `Transition` only when you call `join_with`, so a fan-out without a join does not compile. `fan_out` needs a first target, so handle an empty list first. The showcase returns `Transition::error(..)`, which fails the run with a `TaskError::NodeError`. With branches known up front, write them out. This example starts two branches on two different nodes: ```rust // The fan-out transition defines the branch group and the join that waits for that group. task.register_transition(start, move |input| { Transition::fan_out(&increment, input) .and(&double, input) .join_with(join.join()) })?; // Each branch still declares where its own output should go. Branches can do any amount of work // before eventually transitioning to the join. task.register_transition(increment, join.join())?; task.register_transition(double, join.join())?; task.register_transition(join, task.transitions_to_finish())?; ``` A branch can pass through intermediate nodes before it joins. Only the last node on the branch registers the join target. If a branch joins a different node than the one its fan-out named, the run fails with `TaskError::InvalidState`. ## Join node A join node takes `JoinInput` as its input, where `T` is the payload every branch contributes. `NodeId::join()` exists only on nodes with that input type. `JoinInput` has three methods: * `iter()` yields the payloads in the order the branches were created, not the order they finished. * `len()` returns the number of payloads. * `is_empty()` returns true when there are none. The showcase relies on that order: the prompt lists the crates as `discover` found them, although `swiftide-core` finished last after a retry. A branch whose output does not convert into `T` does not compile. ## Payload mapping A branch does not have to return the payload type itself. `JoinTarget::map` converts the branch output on its way into the join: ```rust let mut task = Task::::new(); let start = task.register_node_fn(|text: &String| -> Result { Ok(text.trim().to_string()) }); let count = task.register_node_fn(|text: &String| -> Result { Ok(text.split_whitespace().count()) }); let length = task .register_node_fn(|text: &String| -> Result { Ok(text.chars().count()) }); let shout = task .register_node_fn(|text: &String| -> Result { Ok(text.to_uppercase()) }); let join = task.register_node_fn(|parts: &JoinInput| -> Result { Ok(parts.iter().cloned().collect::>().join("; ")) }); task.starts_with(start); task.register_transition(start, move |text: String| { Transition::fan_out(&count, text.clone()) .and(&length, text.clone()) .and(&shout, text) .join_with(join.join()) })?; // `count` returns a usize; `map` turns it into the join's String payload. task.register_transition( count, join.join().map(|words: usize| format!("{words} words")), )?; // An async mapping is registered with `register_transition_async`. task.register_transition_async( length, join.join() .map_async(|chars: usize| async move { format!("{chars} characters") }), )?; // `shout` already returns a String, so it joins as is. task.register_transition(shout, join.join())?; task.register_transition(join, task.transitions_to_finish())?; // "2 words; 11 characters; HELLO TASKS" let TaskRunOutcome::Completed(summary) = task.run(" hello tasks ").await? else { anyhow::bail!("nothing in this task pauses"); }; ``` `map` turns the `usize` from `count` into the `String` the join expects. `JoinTarget::map_async` does the same with an async closure and is registered with `Task::register_transition_async`. `shout` already returns a `String`. Any output that converts into the payload type with `Into` needs no mapping. The task returns `"2 words; 11 characters; HELLO TASKS"`, in branch creation order. ## Mixed payloads When branches return different types and the join needs all of them, give the join node an `AnyJoinInput`. Use `join_any()` in place of `join()`, both in the fan-out and for each branch: ```rust let mut task = Task::::new(); let start = task.register_node_fn(|text: &String| -> Result { Ok(text.trim().to_string()) }); let count = task.register_node_fn(|text: &String| -> Result { Ok(text.split_whitespace().count()) }); let shout = task .register_node_fn(|text: &String| -> Result { Ok(text.to_uppercase()) }); let join = task.register_node_fn(|parts: &AnyJoinInput| -> Result { let words: usize = parts.iter::().sum(); let shouted = parts.iter::().cloned().collect::>(); Ok(format!("{} ({words} words)", shouted.join(" "))) }); task.starts_with(start); task.register_transition(start, move |text: String| { Transition::fan_out(&count, text.clone()) .and(&shout, text) .join_with(join.join_any()) })?; task.register_transition(count, join.join_any())?; task.register_transition(shout, join.join_any())?; task.register_transition(join, task.transitions_to_finish())?; // "HELLO TASKS (2 words)" let TaskRunOutcome::Completed(summary) = task.run(" hello tasks ").await? else { anyhow::bail!("nothing in this task pauses"); }; ``` `AnyJoinInput::iter::()` downcasts each payload and skips the ones of other types. `iter_any()` yields every payload as a type-erased value. The compiler no longer checks that a branch delivers a type the join reads, so prefer `JoinInput` with `map` where you can. ## Concurrency model `ConcurrencyModel` has two variants. `Sequential` is the default, and `Parallel` is opt-in. Set the default for the whole task with `with_default_concurrency_model`: ```rust let mut task = Task::new().with_default_concurrency_model(ConcurrencyModel::Parallel); let start = task.register_node_fn(|input: &i32| -> std::result::Result { Ok(*input) }); let increment = task .register_node_fn(|input: &i32| -> std::result::Result { Ok(*input + 1) }); let double = task .register_node_fn(|input: &i32| -> std::result::Result { Ok(*input * 2) }); let join = task.register_node_fn( |input: &JoinInput| -> std::result::Result { Ok(input.iter().copied().sum::()) }, ); task.starts_with(start); ``` To override the model from a given step on, call `concurrency_model` on a `Transition`, `MarkedTransition` or `FanOutTransition`. A branch keeps the override for every later step until another transition overrides it, and the branches of a fan-out inherit its model. The join node always runs with the task’s default model. Under `Sequential`, the runtime evaluates one node at a time. Fan-out branches take turns: after a node finishes, the branch’s next step goes to the back of the queue. Under `Parallel`, the branches run concurrently inside the future that `run` returns, as the [overview](/tasks/overview/#crate-contents) explains. Waiting on input/output (I/O), such as model calls, overlaps. Synchronous, compute-only closures do not. This is the showcase’s captured run, with `claude-sonnet-5-5` as the model for all ten analysts. The `model` and `discover` lines before it and the render steps after it are left out: ```text [ 0.0s] analyse started swiftide-core [ 0.0s] analyse started swiftide-langfuse [ 0.0s] analyse started swiftide-macros [ 0.0s] analyse started swiftide-indexing [ 0.0s] analyse started swiftide-integrations [ 0.0s] analyse started swiftide-query [ 0.0s] analyse started swiftide-tasks [ 0.0s] analyse started swiftide-agents [ 0.0s] analyse started swiftide-agents-contrib [ 0.0s] analyse started swiftide [ 14.0s] analyse finished swiftide-langfuse in 14.0s (6 tool calls) [ 14.3s] analyse finished swiftide-macros in 14.3s (6 tool calls) [ 14.7s] analyse finished swiftide in 14.7s (5 tool calls) [ 19.2s] analyse finished swiftide-query in 19.1s (10 tool calls) [ 19.7s] analyse retry swiftide-agents-contrib: the `stop` arguments do not match the schema: unknown field `arguments`, expected one of `purpose`, `key_types`, `depends_on`, `notes` [ 20.2s] analyse finished swiftide-agents in 20.2s (10 tool calls) [ 24.4s] analyse finished swiftide-indexing in 24.4s (10 tool calls) [ 24.9s] analyse finished swiftide-tasks in 24.9s (10 tool calls) [ 27.1s] analyse finished swiftide-agents-contrib in 27.1s (11 tool calls) [ 31.4s] analyse finished swiftide-integrations in 31.4s (13 tool calls) [ 33.9s] analyse retry swiftide-core: the `stop` arguments do not match the schema: unknown field `arguments`, expected one of `purpose`, `key_types`, `depends_on`, `notes` [ 40.5s] analyse finished swiftide-core in 40.5s (15 tool calls) [ 40.5s] join 10 summaries; 230s of agent work in 41s wall-clock (5.7x) ``` All ten branches start at 0.0s, and the fan-out takes as long as its slowest branch. The join line sums the agent time of every branch: 230 seconds of work in 41 seconds. `OUTPUT.md` estimates close to four minutes for the same analyses run one after another. ## Limits * A branch cannot start a second fan-out while it belongs to an active join group. The run fails with `TaskError::InvalidState`. Fan out again from the join node instead. * A branch cannot finish the task on its own. If a branch inside a group registers `task.transitions_to_finish()`, the task output counts as an arrival at the join, unmapped. When that type differs from the join’s payload type, the join fails at runtime with `TaskError::InvalidState`. Route every branch to the join and finish from there. * The first branch that fails fails the run. The other in-flight branches are dropped and never complete. * All branches share one node instance; keep per-branch state inside `evaluate` ([Agents as task nodes](/tasks/agents-as-nodes/#custom-agent-node)). ## Reference * [`Transition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html) * [`Transition::error`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html#method.error) * [`FanOutTransition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.FanOutTransition.html) * [`MarkedTransition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.MarkedTransition.html) * [`JoinInput`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.JoinInput.html) * [`JoinTarget`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.JoinTarget.html) * [`MappedJoinTarget`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.MappedJoinTarget.html) * [`AnyJoinInput`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.AnyJoinInput.html) * [`AnyJoinTarget`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.AnyJoinTarget.html) * [`ConcurrencyModel`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.ConcurrencyModel.html) * [`NodeId`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeId.html) * [`NodeId::join_any`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeId.html#method.join_any) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html) * [`Task::register_transition_async`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html#method.register_transition_async) * [`TaskError`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskError.html) ## Next [Pause and resume](/tasks/pause-and-resume/) # Nodes and transitions > Register closures and TaskNode types as nodes and connect them with typed transitions from start to finish. Here you build a linear task from two closures and a node type of your own, run it, and find out which node failed when something goes wrong. Import the task types from `swiftide::tasks`. [Tasks overview](/tasks/overview/#feature-flag) shows the feature flag to turn on. ```rust use std::num::ParseIntError; use swiftide::{ reexports::async_trait::async_trait, tasks::{DynNodeId, NodeError, Task, TaskError, TaskNode, TaskRunOutcome, Transition}, }; ``` Register the nodes, connect them, and run the task: ```rust let mut task: Task = Task::new(); // A sync closure node. `Ok::<_, NodeError>` names the error type. let trim = task.register_node_fn(|text: &String| Ok::<_, NodeError>(text.trim().to_string())); // A node type that implements `TaskNode`. let count = task.register_node(CountWords); // A closure that returns a pinned, boxed future. Copy what it needs before `async move`. let label = task.register_node_async_fn(|words: &usize| { let words = *words; Box::pin(async move { Ok::<_, NodeError>(format!("{words} words")) }) }); task.starts_with(trim); task.register_transition(trim, move |text: String| { if text.is_empty() { return Transition::error("there is no text to count"); } count.transitions_with(text).into() })?; task.register_transition(count, move |words| label.transitions_with(words))?; task.register_transition(label, task.transitions_to_finish())?; match task.run(" nodes and transitions ").await? { TaskRunOutcome::Completed(summary) => println!("{summary}"), TaskRunOutcome::Paused => println!("paused; call task.resume() to continue"), } ``` The run prints `3 words`. `Task` fixes the types at both ends. Each node declares its own input and output, so a `String` becomes a `usize` and then a `String` again. `register_node_fn` turns a synchronous closure into a node, and `register_node_async_fn` does the same for a closure that returns a pinned, boxed future (`Box::pin(async move { .. })`). `register_node` takes any value that implements `TaskNode`. All three return a `NodeId`, the typed handle you wire the node with. `NodeId` is `Copy`, so you move it into transition closures without cloning. `id()` returns its number, assigned in registration order from 0. `starts_with` picks the start node, and `register_transition` says where to go after a node. `transitions_with` moves on to another node with a value, and `transitions_to_finish` hands the task its output. Inside a fan-out branch, that output goes to the join instead of ending the task. `run` returns a `TaskRunOutcome`. `Completed` carries the output. `Paused` is covered in [Pause and resume](/tasks/pause-and-resume/). ## Task nodes A task node is a type that implements `TaskNode`: ```rust /// Counts the words in a text. #[derive(Clone, Debug)] pub struct CountWords; #[async_trait] impl TaskNode for CountWords { type Input = String; type Output = usize; type Error = std::convert::Infallible; async fn evaluate( &self, _node_id: &DynNodeId, text: &Self::Input, ) -> Result { Ok(text.split_whitespace().count()) } } ``` `Input` is what the node receives and `Output` is what it returns. `Error` can be any `std::error::Error + Send + Sync + 'static`; `CountWords` cannot fail, so it uses `Infallible`. `evaluate` borrows both the node and the input. The `&DynNodeId` argument is the node’s own id with its concrete type erased, and most nodes ignore it. Write the impl with `#[async_trait]`, which Swiftide re-exports as `swiftide::reexports::async_trait`. `Input` and `Output` must implement `NodeArg`, which covers every type that is `Send + Sync + Debug + Clone + 'static`. A missing derive shows up as an unmet `NodeArg` bound. `register_node` also needs the node itself to be `Clone`. A run does not clone your node. Every evaluation calls the same instance through `&self`, so keep per-run state out of the struct. ## Closure nodes The workspace-architect showcase registers its nodes in one block. `discover` and `write` are closure nodes: examples/workspace-architect/src/graph.rs ```rust let discover = task.register_node_fn({ let progress = progress.clone(); move |root: &WorkspaceRoot| { let members = workspace::discover(root, settings.selection)?; progress.step("discover", format_args!("{members} in {root}")); Ok::<_, workspace::DiscoveryError>(members) } }); let analyse = task.register_node(CrateAnalyst::new(llm.clone(), progress.clone())); let join = task.register_node_fn({ let progress = progress.clone(); move |summaries: &JoinInput| { progress.joined(summaries.len()); architecture_prompt(summaries) } }); // `Arc` is a task node out of the box: `Prompt` in, `String` out. let render = task.register_node(Arc::new(llm.clone()) as Arc); let review = task.register_node(ApprovalGate::new(settings.pre_approved, progress.clone())); let write = task.register_node_async_fn({ let output = settings.output; let progress = progress.clone(); move |draft: &ArchitectureDraft| { let output = output.clone(); let progress = progress.clone(); Box::pin(async move { let written = draft.write_to(&output).await?; progress.step( "write", format_args!("{} ({} bytes)", written.path.display(), written.bytes), ); Ok::<_, std::io::Error>(written) }) } }); ``` * `discover` is a sync closure. `Ok::<_, workspace::DiscoveryError>(members)` pins its error type, so the `?` inside converts into it. * `write` is a plain closure that returns `Box::pin(async move { .. })`. The other four nodes have their own pages: [Agents as task nodes](/tasks/agents-as-nodes/), [Fan-out and join](/tasks/fan-out-and-join/), [Built-in nodes](/tasks/built-in-nodes/) and [Pause and resume](/tasks/pause-and-resume/). A closure node needs a concrete error type. Name it with `Ok::<_, E>(..)` or a return type such as `-> Result`, unless the closure already returns a typed `Result`, as `str::parse` does. The closure is `Fn` and runs once per evaluation, so each future needs its own copy of the captures. That is why `write` clones `output` and `progress` before `async move`. The future may still borrow the input, here `draft`. For reusable domain logic, implement `TaskNode` instead. ## Transitions `examples/tasks.rs` chains a prompt node, an agent node and a second prompt node: ```rust // Tasks are typed from input to output. This task starts with a `Prompt` and finishes with the // final teammate-facing hand-off note. let mut task: Task = Task::new(); // Convenience helper: `Arc` already implements `TaskNode`, so prompt-model // steps can be registered directly without any wrapper code. let prompt_step: Arc = Arc::new(openai.clone()); let brief_prompt_id = task.register_node(prompt_step.clone()); let handoff_prompt_id = task.register_node(prompt_step); // Custom node: the agent step owns the stop-tool contract and returns structured task data. let agent_id = task.register_node(BriefingAgent::new(agent)); task.starts_with(brief_prompt_id); // Step 1: turn the incoming request into a short brief that the agent can work from. task.register_transition(brief_prompt_id, move |brief| { agent_id.transitions_with(brief) })?; // Step 2: use the structured stop payload to drive a final LLM rendering step. task.register_transition(agent_id, move |decision: BriefingDecision| { Transition::next( &handoff_prompt_id, Prompt::from( "Write a teammate-facing hand-off note.\n\ Audience: {{audience}}\n\ Summary: {{summary}}\n\ Next steps:\n\ {% for step in next_steps %}- {{ step }}\n{% endfor %}", ) .with_context_value("audience", decision.audience) .with_context_value("summary", decision.summary) .with_context_value("next_steps", decision.next_steps), ) })?; // Step 3: the rendered hand-off note becomes the task output. task.register_transition(handoff_prompt_id, task.transitions_to_finish())?; ``` A transition closure receives the node’s output by value and returns where to go next. `agent_id.transitions_with(brief)` is shorthand for `Transition::next(&agent_id, brief)`. Both return a `MarkedTransition` that carries the target’s type, so the compiler checks the value against the target’s `Input`. Each node has exactly one transition, and registering a second one returns `TaskError::InvalidState`. To branch, return different transitions from that one closure. When the arms point at nodes of different types, or one arm pauses or fails, convert each `MarkedTransition` into a `Transition` with `.into()`, as the linear task does. `Transition::error` fails the whole run, drops any in-flight fan-out branches and leaves nothing to resume. It accepts anything that converts into `Box`, including a `&str`. `Transition::pause()` pauses the branch. When the decision needs to await something, use `register_transition_async`. Its closure returns a future, such as `move |value| async move { next.transitions_with(value) }`. ## Start and finish The start node’s `Input` must be the task’s `Input`, or `starts_with` does not compile. `run` takes `impl Into`, so the linear task can pass a `&str` to a `Task`. `transitions_to_finish()` returns a closure that takes the task’s `Output`. The finish constructor on `Transition` is private, so the compiler checks the value against that task’s output type. A finish closure taken from a different task still compiles, and `run` then fails with `TaskError::TypeError`. Code from 0.32 called `transitions_to_done` and got an `Option` from `run`. [Upgrading](/reference/upgrading/#tasks-crate) lists the 0.33 changes. ## Errors Match on the result of `run` to tell a failing node from a wiring mistake: ```rust let mut task: Task = Task::new(); let parse = task.register_node_fn(|text: &String| text.trim().parse::()); task.starts_with(parse); task.register_transition(parse, task.transitions_to_finish())?; match task.run("forty-two").await { Ok(TaskRunOutcome::Completed(number)) => println!("parsed {number}"), Ok(TaskRunOutcome::Paused) => println!("paused"), Err(TaskError::NodeError(failed)) => { // `node_error` is the error the node returned, boxed. if let Some(error) = failed.node_error.downcast_ref::() { eprintln!("node {} could not parse the input: {error}", failed.node_id); } } Err(wiring @ (TaskError::NoSteps | TaskError::MissingTransition(_))) => { // `run` checks the graph before the first node runs. panic!("the task is wired wrong: {wiring}"); } Err(other) => return Err(other.into()), } ``` A failing node makes `run` return `TaskError::NodeError`. The `NodeError` inside carries `node_id`, the number of the failing node, and `node_error`, the node’s own error boxed. Downcast `node_error` to read your error type. The error displays as `Node error in node 0: invalid digit found in string`. A `Transition::error` produces the same variant, with the id of the node whose transition failed. `run` and `resume` check the graph before any node runs. Without a start node you get `TaskError::NoSteps`. When any registered node has no transition, you get `TaskError::MissingTransition`, even when no path reaches that node, with a message such as `Node 3 is missing a transition`. `TaskError::Incomplete` means all work drained without a finish or a pause. ## Reference * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html) * [`TaskNode`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.TaskNode.html) * [`NodeId`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeId.html) * [`DynNodeId`](https://docs.rs/swiftide/0.34/swiftide/tasks/type.DynNodeId.html) * [`NodeArg`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.NodeArg.html) * [`Transition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html) * [`MarkedTransition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.MarkedTransition.html) * [`TaskRunOutcome`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskRunOutcome.html) * [`TaskError`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskError.html) * [`NodeError`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeError.html) ## Next [Fan-out and join](/tasks/fan-out-and-join/) # How tasks work > A task is a typed graph of nodes and transitions that runs functions, prompts and agents, with fan-out, join and pause. You will be able to read a task graph, enable the `swiftide-tasks` feature flag, and decide whether a job needs a task or a single agent. A task runs a fixed graph of steps, and each step can be a function, a prompt or an agent. [Your first task](/start/your-first-task/) builds a two-node task step by step if you want a smaller start. ## Showcase graph The workspace-architect showcase drafts an `ARCHITECTURE.md` for a Cargo workspace. This function builds its whole task: examples/workspace-architect/src/graph.rs ```rust /// Builds the task: `WorkspaceRoot` in, the written ARCHITECTURE.md out. pub fn architect_task( llm: &L, settings: ArchitectSettings, progress: &Progress, ) -> Result, TaskError> where L: ChatCompletion + SimplePrompt + Clone + 'static, { let mut task = Task::new(); let discover = task.register_node_fn({ let progress = progress.clone(); move |root: &WorkspaceRoot| { let members = workspace::discover(root, settings.selection)?; progress.step("discover", format_args!("{members} in {root}")); Ok::<_, workspace::DiscoveryError>(members) } }); let analyse = task.register_node(CrateAnalyst::new(llm.clone(), progress.clone())); let join = task.register_node_fn({ let progress = progress.clone(); move |summaries: &JoinInput| { progress.joined(summaries.len()); architecture_prompt(summaries) } }); // `Arc` is a task node out of the box: `Prompt` in, `String` out. let render = task.register_node(Arc::new(llm.clone()) as Arc); let review = task.register_node(ApprovalGate::new(settings.pre_approved, progress.clone())); let write = task.register_node_async_fn({ let output = settings.output; let progress = progress.clone(); move |draft: &ArchitectureDraft| { let output = output.clone(); let progress = progress.clone(); Box::pin(async move { let written = draft.write_to(&output).await?; progress.step( "write", format_args!("{} ({} bytes)", written.path.display(), written.bytes), ); Ok::<_, std::io::Error>(written) }) } }); task.starts_with(discover); // One branch per crate, all evaluating the same analyst node with a different input. task.register_transition(discover, move |members: WorkspaceMembers| { let mut crates = members.crates.into_iter(); let Some(first) = crates.next() else { return Transition::error("the workspace has no crates to document"); }; crates .fold(Transition::fan_out(&analyse, first), |fan_out, member| { fan_out.and(&analyse, member) }) .concurrency_model(ConcurrencyModel::Parallel) .join_with(join.join()) })?; // Each branch hands its `CrateSummary` to the join; the join runs once all have arrived. task.register_transition(analyse, join.join())?; task.register_transition(join, { let progress = progress.clone(); move |prompt| { progress.step("render", "drafting ARCHITECTURE.md"); render.transitions_with(prompt) } })?; task.register_transition(render, move |markdown: String| { review.transitions_with(ArchitectureDraft::from_model_output(&markdown)) })?; task.register_transition(review, move |decision| match decision { Review::Approved(draft) => write.transitions_with(draft).into(), Review::AwaitingApproval => Transition::pause(), })?; task.register_transition(write, task.transitions_to_finish())?; Ok(task) } ``` The task has the type `Task`. It takes a workspace root and returns the written document. Each `register_node*` call adds a node and returns its typed `NodeId`. Each `register_transition` call says where a node’s output goes next. * **discover** is a closure node. It runs `cargo metadata` and returns the member crates. * **analyse** is a custom task node, `CrateAnalyst`. The fan-out sends every crate to it as its own branch. The fan-out sets `ConcurrencyModel::Parallel`, so the branches run at the same time. Each branch runs a fresh agent that reads one crate and returns a `CrateSummary`. * **join** waits for every branch. It receives a `JoinInput` and builds the prompt for the writer. * **render** is an `Arc` registered as a node. It sends the prompt to the model and returns Markdown. * **review** is `ApprovalGate`. Until someone approves the draft, its transition returns `Transition::pause()` and the run stops. * **write** is an async closure node. It saves the file, and its transition, `task.transitions_to_finish()`, ends the task. In the captured run, ten analyst agents on `claude-sonnet-5-5` finished in the time of the slowest one. The join line from `OUTPUT.md`: ```text [ 40.5s] join 10 summaries; 230s of agent work in 41s wall-clock (5.7x) ``` Run it against the Swiftide workspace from the repository root: ```sh export OPENAI_API_KEY=sk-... cargo run -p workspace-architect ``` The command-line program starts the task with `task.run(WorkspaceRoot)`. That returns `TaskRunOutcome::Completed(WrittenDocument)` or `TaskRunOutcome::Paused`: examples/workspace-architect/src/main.rs ```rust let mut outcome = task.run(root).await?; loop { match outcome { TaskRunOutcome::Completed(written) => return Ok(Some(written)), TaskRunOutcome::Paused => { if !approve_paused_draft(&task)? { return Ok(None); } outcome = task.resume().await?; } } } ``` On a pause, the program asks you to approve the draft and then calls `resume()`. ## Task graph ![Task graph](/diagrams/diagram-task-graph.svg) The diagram shows a smaller task with the same parts. A start node fans out to two prompt nodes that run in parallel. A join node collects both outputs, and the task finishes. A task has these parts: * A **task node** implements `TaskNode`. It declares an `Input`, an `Output` and an `Error` type, and its `evaluate` method turns a borrowed input into an output. [Nodes and transitions](/tasks/nodes-and-transitions/#task-nodes) covers the trait. * A **transition** decides where a node’s output goes next. It is a closure that takes the output by value, or a ready-made target such as `join.join()` or `task.transitions_to_finish()`. Each node has exactly one transition. * The **start node** is set with `starts_with`. Its input type is the task’s input type. * A **fan-out** sends one input to each of several branches. Every fan-out names the join node that waits for all of them. By default the branches run one after another. Set `ConcurrencyModel::Parallel` on the fan-out or the task to run them at the same time. * A **join** node receives the branch outputs as one `JoinInput`. [Fan-out and join](/tasks/fan-out-and-join/) covers both. * A **pause** stops the run, and `run` returns `TaskRunOutcome::Paused`. The paused branch keeps the node whose transition paused. `resume()` runs that node again with the same input. This is why the showcase pauses on the cheap review node and not on render. [Pause and resume](/tasks/pause-and-resume/) covers the details. When a branch reaches the finishing transition, `run` or `resume()` returns `TaskRunOutcome::Completed` with the task’s output. [Nodes and transitions](/tasks/nodes-and-transitions/#errors) covers the checks `run` makes before any node runs. ## Crate contents The `swiftide-tasks` crate holds the graph and its runtime: * `Task`, `TaskNode`, `NodeId` and `Transition` * fan-out and join, with `JoinInput` and `AnyJoinInput` * `ConcurrencyModel::Sequential` (the default) and `ConcurrencyModel::Parallel` * pause, resume and reset, with `TaskRunOutcome` * `TaskError` and `NodeError` * closure nodes through `register_node_fn` and `register_node_async_fn` * [built-in nodes](/tasks/built-in-nodes/) for `SimplePrompt`, `ChatCompletion` and `ToolExecutor` trait objects The crate’s own code spawns no tasks or threads. Parallel branches run concurrently inside the future that `run` returns. Swiftide itself still depends on tokio. `TaskAgent`, which wraps an agent as a node, lives in swiftide-agents. Import it from `swiftide::agents::TaskAgent`. ## Feature flag Turn on the `swiftide-tasks` feature flag: ```sh cargo add swiftide@0.34 --features swiftide-tasks ``` The `swiftide-agents` flag turns it on as well. Import the graph types from `swiftide::tasks`. The old path `swiftide::agents::tasks` still works, but it is deprecated since 0.32.1. [Upgrading](/reference/upgrading/#tasks-crate) lists the 0.33 changes to tasks. A comment in the crate source says the tasks application programming interface (API) “is not stable yet” and “is subject to rapid change”, so check Upgrading when you move to a new minor version. ## Tasks and agents An [agent](/agents/overview/) runs a loop. After every model response it decides which tool to call next. It stops when a tool or the model asks it to, when it reaches its completion limit, or when it fails. A task fixes the order of steps in code. Every node declares its input and output types. A transition to a node with the wrong input type does not compile, and neither does a fan-out without a join. Control flow lives in Rust closures, not in model output. The model can still make decisions inside a node, but the graph decides what happens with the result. Each node does one job. You can test a closure node without a model, swap the model behind a prompt node, or give an agent node its own tools and limits. Use a task when you know the steps in advance, when some of them need no model, or when you want to fan out work and collect it in one place. Put an agent inside a node for the open-ended part, such as reading one crate, and let the task handle the order, the parallel branches and the approval. [Agents as task nodes](/tasks/agents-as-nodes/) shows how. The [workspace-architect showcase](/showcases/workspace-architect/) walks through the whole program. ## Reference * [`swiftide::tasks`](https://docs.rs/swiftide/0.34/swiftide/tasks/index.html) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html) * [`TaskNode`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.TaskNode.html) * [`NodeId`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeId.html) * [`Transition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html) * [`JoinInput`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.JoinInput.html) * [`AnyJoinInput`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.AnyJoinInput.html) * [`ConcurrencyModel`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.ConcurrencyModel.html) * [`TaskRunOutcome`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskRunOutcome.html) * [`TaskError`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskError.html) * [`NodeError`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeError.html) * [`TaskAgent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.TaskAgent.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [Feature flags](/reference/feature-flags/) * [Crates](/reference/crates/) ## Next [Nodes and transitions](/tasks/nodes-and-transitions/) # Pause and resume > Pause a task at an approval gate, inspect the paused node, and resume or rerun it with the same Task value. You end up with a task that stops before a side effect, waits for a person or an outside event, and then continues from the node where it stopped. The workspace-architect showcase uses this to hold its generated `ARCHITECTURE.md` until someone approves it: examples/workspace-architect/src/graph.rs ```rust task.register_transition(review, move |decision| match decision { Review::Approved(draft) => write.transitions_with(draft).into(), Review::AwaitingApproval => Transition::pause(), })?; ``` The `review` node returns a `Review`. When the draft is approved, the transition sends it on to the `write` node. When it still waits for approval, the transition returns `Transition::pause()`, and `run` comes back with `TaskRunOutcome::Paused` instead of an output. The `.into()` on the first arm is the same `MarkedTransition` conversion described in [Nodes and transitions](/tasks/nodes-and-transitions/#transitions). The types live in `swiftide::tasks`, behind the `swiftide-tasks` [feature flag](/tasks/overview/#feature-flag): ```sh cargo add swiftide@0.34 --features swiftide-tasks ``` A task pause is not the same as an agent pause. To stop an agent until someone approves a single tool call, see [Human in the loop](/agents/human-in-the-loop/). ## Approval gate The `review` node is `ApprovalGate`, a task node that remembers the draft and whether someone approved it: examples/workspace-architect/src/approval.rs ```rust /// Lets a draft through once it is approved, and otherwise keeps it for review. /// /// The graph pauses the task when the gate answers [`Review::AwaitingApproval`]. A paused branch /// re-runs its node on `resume()`, so after [`ApprovalGate::approve`] the same draft passes /// through. The gate is cheap to evaluate, which is why the pause sits here and not on the /// expensive render node. #[derive(Debug, Clone)] pub struct ApprovalGate { state: Arc>, progress: Progress, } #[derive(Debug, Default)] struct GateState { approved: bool, pending: Option, } impl ApprovalGate { pub fn new(pre_approved: bool, progress: Progress) -> Self { let state = GateState { approved: pre_approved, pending: None, }; Self { state: Arc::new(Mutex::new(state)), progress, } } /// The draft waiting for a decision, if any. pub fn pending_draft(&self) -> Option { self.state().pending.clone() } pub fn approve(&self) { self.state().approved = true; } fn state(&self) -> MutexGuard<'_, GateState> { self.state.lock().unwrap_or_else(PoisonError::into_inner) } } #[async_trait] impl TaskNode for ApprovalGate { type Input = ArchitectureDraft; type Output = Review; type Error = Infallible; async fn evaluate( &self, _node_id: &DynNodeId, draft: &ArchitectureDraft, ) -> Result { let mut state = self.state(); if state.approved { state.pending = None; self.progress.step("review", "draft approved"); return Ok(Review::Approved(draft.clone())); } state.pending = Some(draft.clone()); self.progress.step( "review", format_args!( "draft ready ({} bytes), waiting for approval", draft.as_str().len() ), ); Ok(Review::AwaitingApproval) } } ``` A paused branch keeps its current node and that node’s input. On `resume()`, the runtime evaluates the same node again with the same input, and then runs its transition again. The gate relies on this. The first evaluation stores the draft and answers `AwaitingApproval`. After `approve()`, the second evaluation answers `Approved` with the same draft. Put the pause on a cheap node. The showcase could pause right after `render`, but then `resume()` would call the model and draft the document a second time. The gate keeps its state in `Arc>`. The `Mutex` is there because `evaluate` and `approve` take `&self`, so plain fields cannot change. The `Arc` is there because a task node must be `Clone`, and `Mutex` is not. With the `Arc`, clones of the gate share one state. The gate’s error type is `Infallible`, because waiting is an answer, not a failure. ## Resume loop The command-line interface (CLI) runs the task and loops until it completes or the user declines the draft: examples/workspace-architect/src/main.rs ```rust let mut outcome = task.run(root).await?; loop { match outcome { TaskRunOutcome::Completed(written) => return Ok(Some(written)), TaskRunOutcome::Paused => { if !approve_paused_draft(&task)? { return Ok(None); } outcome = task.resume().await?; } } } ``` On `TaskRunOutcome::Paused`, `approve_paused_draft` (in the same file) finds the gate with `task.current_nodes::()`, shows the draft and asks for confirmation. On yes it calls `gate.approve()` and returns `true`, and the loop calls `task.resume()`. On no it returns `false`, and `draft_architecture` returns `Ok(None)` and drops the paused task. Nothing is written. This is the end of a captured run, with `claude-sonnet-5-5` as the writer. The draft between the review line and the question is trimmed: ```text [ 68.8s] review draft ready (11210 bytes), waiting for approval ... Write this ARCHITECTURE.md? [y/N] [ 68.8s] review draft approved [ 68.8s] write .../target/ARCHITECTURE.md (11210 bytes) Wrote .../target/ARCHITECTURE.md (11210 bytes) in 68.8s. ``` The `--yes` flag creates the gate already approved, so the first evaluation answers `Approved` and the task never pauses. ## Current nodes `Task::current_nodes::()` gives you the paused node, so you can read or change its state before you resume. This helper delivers the CI result from the next section through the task: ```rust /// Delivers a CI result through the task, for callers that hold the task instead of the node. pub fn deliver_through_task(task: &Task, result: &str) -> anyhow::Result<()> { let CurrentNodes::One(ci) = task.current_nodes::() else { anyhow::bail!("the task is not waiting on exactly one CI result"); }; ci.deliver(result); Ok(()) } ``` The method returns the current nodes of type `T` as a `CurrentNodes`: * `CurrentNodes::None` when no current node has that type. * `CurrentNodes::One(&T)` for exactly one. * `CurrentNodes::Multiple(Vec<&T>)` for more than one. Paused branches take precedence. The method returns runnable nodes only when no branch is paused. In a parallel fan-out several branches can pause on the same node type, so handle `Multiple` there. The references point at the registered node instances. ## External callbacks The same pattern waits for another system, such as a continuous integration (CI) webhook. The node reads a slot that the webhook handler fills, and the transition pauses while the slot is empty: ```rust /// Waits for a CI result that a webhook delivers from outside the task. #[derive(Debug, Clone, Default)] pub struct AwaitCi { result: Arc>>, } impl AwaitCi { /// Called by the webhook handler. Clones share the slot, so any clone can deliver. pub fn deliver(&self, result: impl Into) { let mut slot = self.result.lock().unwrap_or_else(PoisonError::into_inner); *slot = Some(result.into()); } } #[async_trait] impl TaskNode for AwaitCi { type Input = String; type Output = CiStatus; type Error = Infallible; async fn evaluate( &self, _node_id: &DynNodeId, commit: &String, ) -> Result { let slot = self.result.lock().unwrap_or_else(PoisonError::into_inner); Ok(match slot.as_deref() { Some(result) => CiStatus::Finished(format!("{commit}: {result}")), None => CiStatus::Pending, }) } } pub async fn wait_for_ci() -> anyhow::Result { let ci = AwaitCi::default(); let mut task = Task::::new(); let push = task .register_node_fn(|commit: &String| -> Result { Ok(commit.clone()) }); let wait = task.register_node(ci.clone()); task.starts_with(push); task.register_transition(push, move |commit| wait.transitions_with(commit))?; task.register_transition(wait, { let finish = task.transitions_to_finish(); move |status| match status { CiStatus::Finished(report) => finish(report), CiStatus::Pending => Transition::pause(), } })?; assert_eq!(task.run("4f2a91c").await?, TaskRunOutcome::Paused); // One task value holds one run: a second `run` while paused is refused. assert!(matches!( task.run("77b0e3d").await, Err(TaskError::TaskActive) )); // Later, the webhook handler that holds `ci` receives the result, // and the same task value continues. ci.deliver("all checks passed"); let TaskRunOutcome::Completed(report) = task.resume().await? else { anyhow::bail!("the CI result was delivered, so the task should finish"); }; ``` The excerpt registers `ci.clone()` and keeps `ci` itself. In a server, `ci` goes to the webhook handler, and the comment marks where that handler runs. The clone shares the `Arc`, so `deliver` fills the slot that the registered node reads. A handler that holds the task instead can use `deliver_through_task` from the previous section. Keep the same `Task` value alive between the pause and the resume. The runtime state is not serializable. If the process restarts, the paused run is gone, so store what you need to start again, such as the commit. In a server, keep paused tasks in a map keyed by job. One task value holds one run. While the task has paused or queued work, `run` returns `TaskError::TaskActive`. Build a new task for the next job, for example with a function like the showcase’s `architect_task`. ## Reruns after reset `reset()` discards the current run and queues a new one at the start node, with the last input passed to `run`. A `resume()` after it runs that queued run: ```rust let mut task = Task::::new(); let double = task.register_node_fn(|n: &i32| -> Result { Ok(n * 2) }); task.starts_with(double); task.register_transition(double, task.transitions_to_finish())?; assert_eq!(task.run(21).await?, TaskRunOutcome::Completed(42)); // A completed run leaves nothing to continue. assert!(matches!(task.resume().await, Err(TaskError::NotResumable))); // `reset` queues a new run at the start node with the last input. task.reset(); // That queued run counts as active work, so `run` with another input is refused. assert!(matches!(task.run(5).await, Err(TaskError::TaskActive))); assert_eq!(task.resume().await?, TaskRunOutcome::Completed(42)); // A clone has the same graph and a fresh runtime of its own. let mut copy = task.clone(); assert_eq!(copy.run(5).await?, TaskRunOutcome::Completed(10)); ``` * After a completed run there is nothing to continue, so `resume()` returns `TaskError::NotResumable`. * `reset()` also discards a paused run. Use it to start over with the same input instead of continuing. * The queued run counts as active work, so `run(other_input)` after `reset()` returns `TaskError::TaskActive`. To start over with a different input, use `task.clone()`. Calling `starts_with` again also clears the runtime state. * `reset()` before the first `run` has no input to queue. The `resume()` that follows returns `TaskError::NotResumable`. * A failed run leaves no state to resume. `reset()` and `resume()` retry it with the same input. * `task.clone()` copies the graph with a fresh runtime, without the paused run. Node state behind an `Arc` is shared between the copies. ## Pauses in parallel branches A pause stops the whole task, not only its branch. When a branch asks to pause, the runtime starts no new branches. Parallel branches that are already running finish their current node and transition, and their next steps wait in the queue. Then `run` returns `Paused`, unless one of those running branches reaches the finish transition. In that case `run` returns `Completed` and the paused branch is discarded. If a running branch fails after another branch asked to pause, `run` returns that error. The paused run is lost, and `resume()` has nothing to continue. `resume()` puts the paused branches back in the queue in the order they were created and continues with all queued work. A join runs only when every branch has arrived, so a paused branch also holds back its join. [Fan-out and join](/tasks/fan-out-and-join/#join-node) covers branches and joins. ## Reference * [`Transition::pause`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html#method.pause) * [`MarkedTransition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.MarkedTransition.html) * [`Task::run`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html#method.run) * [`Task::resume`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html#method.resume) * [`Task::reset`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html#method.reset) * [`Task::current_nodes`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html#method.current_nodes) * [`CurrentNodes`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.CurrentNodes.html) * [`TaskRunOutcome`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskRunOutcome.html) * [`TaskError`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskError.html) * [`TaskNode`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.TaskNode.html) ## Next [Agents as task nodes](/tasks/agents-as-nodes/)