Integrations: model providers, vector stores and other integrations # Anthropic > Build an Anthropic client for Claude prompts, chat completions, streaming and agents with tools; it has no embeddings or structured output. You end up with an `Anthropic` client that sends prompts and chat completions to Claude, streams responses and runs agents with tools. Enable the `anthropic` feature flag for the client and `swiftide-agents` for the agent; the `all` bundle leaves both out: ```sh cargo add swiftide@0.34 --features anthropic,swiftide-agents cargo add tokio --features full cargo add anyhow futures-util ``` The snippets use `tokio` and `anyhow` around the calls. The streaming snippet needs `futures_util::StreamExt` for `stream.next()`, which Swiftide does not re-export. ## Client ```rust let anthropic = Anthropic::builder() .default_prompt_model("claude-sonnet-4-6") .on_usage(|usage| { println!( "{} prompt + {} completion tokens", usage.prompt_tokens, usage.completion_tokens ); Ok(()) }) .build()?; let answer = anthropic .prompt("Explain the borrow checker in one sentence.".into()) .await?; ``` `Anthropic::builder()` returns an `AnthropicBuilder`. `default_prompt_model` sets the Claude model for every call. `build()` creates the underlying `async_anthropic::Client`, which reads your key from the `ANTHROPIC_API_KEY` environment variable. A missing key only logs a warning. The first request then fails as unauthorized. The client implements `SimplePrompt` and `ChatCompletion`. You can pass it to indexing transformers that prompt a large language model (LLM), to query pipelines and to agents. `on_usage` receives the token counts of each call. On `prompt` and `complete` the client awaits it, and an `Err` from the callback fails the call. Use `on_usage_async` for a callback that awaits. [Usage callbacks](/models/errors-backoff-and-usage/#usage-callbacks) compares this with the other clients. Swiftide builds the HTTP (Hypertext Transfer Protocol) client with rustls, even when another crate turns on native TLS (Transport Layer Security) for reqwest. It uses the rustls crypto provider your application installed, or else ring, so it needs no setup; see [TLS](/integrations/overview/#tls). ## Models Always set `default_prompt_model`. Without it the client asks for `claude-3-5-sonnet-20241022`, which Anthropic has retired, so every call fails. `Options` holds only `prompt_model`. The client sends no temperature, stop sequences or thinking settings. Every request asks for at most 2048 output tokens, the default of `async-anthropic` 0.6, and Swiftide has no setting to raise it. A longer answer is cut off, and `ChatCompletionResponse` has no field that tells you it was truncated. `async-anthropic` 0.6 parses only text, tool use and tool result content blocks. A response that holds a thinking block fails to parse. If your model returns thinking blocks without being asked, every call fails. The snippet uses `claude-sonnet-4-6`, which does not. ## Agents with tools Define a tool with the `#[swiftide::tool]` macro: ```rust #[swiftide::tool( description = "Count the words in a piece of text", param(name = "text", description = "The text to count") )] async fn count_words(_context: &dyn AgentContext, text: &str) -> Result { Ok(ToolOutput::text( text.split_whitespace().count().to_string(), )) } ``` Then hand the client to the agent: ```rust let mut agent = Agent::builder() .llm(&anthropic) .tools(vec![count_words()]) .limit(5) .build()?; agent .query("How many words are in 'the quick brown fox'? Use the tool.") .await?; ``` `AgentBuilder::llm` clones the client into the agent. Each request carries the tool specs as Anthropic tools with the tool choice set to auto, so the model decides when to call one. The default `stop` tool comes along with your own tools. The client turns the message history into Anthropic messages: * The first system message becomes the request’s `system` field. Later system messages and summaries go out as user messages. * Tool outputs that follow each other go out together in one user message. Swiftide groups them since 0.33. * A tool output without content goes out as `Success`. * `ChatMessage::Reasoning` entries are left out. * A message with content parts may hold text parts only. Any other part, such as an image, returns an error. See [Images and multimodal input](/models/images/). ## Streaming ```rust let request = ChatCompletionRequest::builder() .messages(vec![ ChatMessage::new_system("You answer in plain words."), ChatMessage::new_user("Explain lifetimes in Rust in three sentences."), ]) .build()?; let mut stream = anthropic.complete_stream(&request).await; let mut printed = 0; let mut last = None; while let Some(response) = stream.next().await { let response = response?; let text = response.message().unwrap_or_default(); print!("{}", &text[printed..]); printed = text.len(); last = Some(response); } if let Some(usage) = last.and_then(|response| response.usage) { println!("\n{} tokens", usage.total_tokens); } ``` `complete_stream` returns a `ChatCompletionStream`. Every item is the whole response so far, and the last item is the complete response with `usage` set. Treat streamed `usage` as an estimate. The client adds up the counts from the message start and message delta events. Anthropic reports cumulative counts in the message delta event, so the totals can come out higher than the non-streaming counts for the same request. The client has no `stream_full` setting, so you always receive the accumulated response. `delta.message_chunk` holds the latest text, but it keeps that value on events that carry no text. Print the part of `message` you have not printed yet, as the snippet does, rather than printing every delta. On streams the client spawns the usage callback and only logs an `Err` from it. An agent streams when you add an `on_stream` hook. ## Limitations * No `EmbeddingModel`. Embed with [FastEmbed](/integrations/fastembed/) or [OpenAI](/integrations/openai/#embeddings) and keep Anthropic for prompts and agents. * No `StructuredPrompt`. See [Structured output](/models/structured-output/) for clients that have it. * No `SparseEmbeddingModel`. To reach Claude through AWS (Amazon Web Services) with structured output and reasoning, use [AWS Bedrock](/integrations/aws-bedrock/). ## Errors and retries `async-anthropic` retries a request that gets HTTP 429 (rate limit) or 529 (overloaded) by itself. It waits 15 seconds first, doubles the wait and gives up after 120 seconds. Streaming requests get no such retry. After that Swiftide marks the error permanent, so `LanguageModelWithBackOff` does not retry it again. On `prompt`, a network error or a response body that fails to parse comes back as a `TransientError`. `async-anthropic` reports both as a network error. A response with a thinking block therefore gets retried by `LanguageModelWithBackOff`, and it fails the same way each time. On `complete` and `complete_stream` every error is permanent. See [Backoff](/models/errors-backoff-and-usage/#backoff). ## Custom client `AnthropicBuilder::client` takes your own `async_anthropic::Client`. Swiftide does not re-export the crate, so add it at the version Swiftide uses: ```sh cargo add async-anthropic@0.6 --no-default-features cargo add backoff@0.4 ``` Start from `async_anthropic::Client::from_api_key` to pass a key from your own configuration. Call `async_anthropic::Client::with_backoff` on it to change the retry timing. It takes a `backoff::ExponentialBackoff`, and `async-anthropic` does not re-export the `backoff` crate, hence the second line. `async_anthropic::Client::builder()` leaves the base URL and API version empty, so prefer those two methods. The client does not read a base URL from the environment. ## Reference * [`Anthropic`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.Anthropic.html) * [`AnthropicBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.AnthropicBuilder.html) * [`AnthropicBuilder::client`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.AnthropicBuilder.html#method.client) * [`AnthropicBuilder::default_prompt_model`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.AnthropicBuilder.html#method.default_prompt_model) * [`AnthropicBuilder::on_usage`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.AnthropicBuilder.html#method.on_usage) * [`AnthropicBuilder::on_usage_async`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.AnthropicBuilder.html#method.on_usage_async) * [`Options` (Anthropic)](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.Options.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`ChatCompletionStream`](https://docs.rs/swiftide/0.34/swiftide/traits/type.ChatCompletionStream.html) * [`ChatCompletionRequest`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionRequest.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatCompletionResponseDelta`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponseDelta.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`Usage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.Usage.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder::llm`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.llm) * [`tool` macro](https://docs.rs/swiftide/0.34/swiftide/attr.tool.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) ## Next [AWS Bedrock](/integrations/aws-bedrock/) # AWS Bedrock > Call Claude and other AWS Bedrock models through the Converse API with AwsBedrock, for prompts in indexing pipelines, agents with tools and structured output. This page leaves you with an `AwsBedrock` client that sends requests to Amazon Web Services (AWS) Bedrock through the Converse application programming interface (API), and uses it in an indexing pipeline and in an agent with tools. ```sh cargo add swiftide@0.34 --features aws-bedrock cargo add tokio --features macros,rt-multi-thread ``` examples/aws_bedrock.rs ```rust let aws_bedrock = integrations::aws_bedrock_v2::AwsBedrock::builder() .default_prompt_model("global.anthropic.claude-haiku-4-5-20251001-v1:0") .build()?; let memory_storage = MemoryStorage::default(); ``` ## Client `AwsBedrock` lives in `swiftide::integrations::aws_bedrock_v2` behind the `aws-bedrock` feature flag. Since 0.33 it uses Bedrock’s Converse and ConverseStream APIs, so one client covers every model family that Converse supports. [Upgrading](/reference/upgrading/) covers the move from the old module. `default_prompt_model` takes a model ID or an Amazon Resource Name (ARN). The `global.` prefix in the example is a cross-region inference profile ID, which Bedrock accepts in place of a model ID. Without a model, the first call fails with a permanent `Model not set` error. Building the client does not check it. The client implements `ChatCompletion` (`complete` and `complete_stream`), `SimplePrompt` and `DynStructuredPrompt`. It does not implement `EmbeddingModel`, so pair it with [FastEmbed](/integrations/fastembed/) or another embedder for vectors. ## Credentials and region Without `.client(..)`, the builder loads the AWS configuration with `aws_config::load_from_env`. That is the AWS SDK (software development kit) credential chain: environment variables, shared profiles, single sign-on (SSO) and IAM (Identity and Access Management) roles. Credentials from `aws login` (AWS Console sign-in) are not supported; use a profile, SSO or environment variables, or pass your own client. The region comes from the same chain, for example `AWS_REGION` or your profile. Your account also needs access to the model you name. The builder runs that loading step inside `tokio::task::block_in_place` when you call `build`. So `build` needs a multi-threaded Tokio runtime. `#[tokio::main]` gives you one. `#[tokio::test]` defaults to a current-thread runtime, where `build` panics; use `#[tokio::test(flavor = "multi_thread")]` there. To control the configuration yourself, build an `aws_sdk_bedrockruntime::Client` and pass it to `AwsBedrockBuilder::client`. Swiftide does not re-export the SDK, so add `aws-config` and `aws-sdk-bedrockruntime` to your own crate at the major versions Swiftide uses (1.x). A client you pass in also skips the blocking load, so `build` also works on a current-thread runtime. ## Pipeline summaries examples/aws_bedrock.rs ```rust indexing::Pipeline::from_loader(FileLoader::new("./README.md")) .log_nodes() .then_chunk(transformers::ChunkMarkdown::from_chunk_range(100..512)) .then(transformers::MetadataSummary::new(aws_bedrock.clone())) .then_store_with(memory_storage.clone()) .log_all() .run() .await?; ``` `MetadataSummary` takes any `SimplePrompt`, so `AwsBedrock` works where other examples use `OpenAI`. Each chunk costs one Converse call, so wrap the client in `LanguageModelWithBackOff` for large inputs. `from_chunk_range(100..512)` drops chunks shorter than 100 characters, so short README sections never reach the model or the storage. The upper bound of the range is exclusive. ## Agents with Claude An agent needs the `swiftide-agents` feature flag, and tool arguments need `schemars` and `serde` in your crate: ```sh cargo add swiftide@0.34 --features aws-bedrock,swiftide-agents cargo add schemars@1 serde anyhow tokio --features serde/derive,tokio/macros,tokio/rt-multi-thread ``` Define the tools: examples/aws_bedrock_agent.rs ```rust #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] struct FormatTimestampRequest { /// Prefix to prepend to the timestamp. prefix: String, /// Timestamp to format. timestamp: String, } #[swiftide::tool(description = "Get the current UTC date and time in RFC3339 format")] async fn current_utc_time(context: &dyn AgentContext) -> Result { let command_output = context .executor() .exec_cmd(&Command::shell("date -u +\"%Y-%m-%dT%H:%M:%SZ\"")) .await?; Ok(command_output.into()) } #[swiftide::tool( description = "Format a timestamp with a caller-provided prefix", param(name = "request", description = "Timestamp formatting input") )] async fn format_timestamp( _context: &dyn AgentContext, request: FormatTimestampRequest, ) -> Result { Ok(ToolOutput::text(format!( "{}{}", request.prefix, request.timestamp ))) } ``` The `use` lines sit above the excerpt in the linked example source. They bring in `anyhow::Result`, `schemars::JsonSchema`, serde’s `Serialize` and `Deserialize`, and from Swiftide `agents`, `ToolOutput`, `ToolError`, `AgentContext`, `Command` and `aws_bedrock_v2::AwsBedrock`. Build the client and the agent, then run it: examples/aws_bedrock_agent.rs ```rust let bedrock = AwsBedrock::builder() .default_prompt_model("global.anthropic.claude-sonnet-4-6") .build()?; ``` examples/aws_bedrock_agent.rs ```rust let mut agent = agents::Agent::builder() .llm(&bedrock) .tools(vec![current_utc_time(), format_timestamp()]) .on_new_message(|_, msg| { let rendered = msg.to_string(); Box::pin(async move { println!("{rendered}"); Ok(()) }) }) .limit(6) .build()?; ``` examples/aws_bedrock_agent.rs ```rust agent .query( "Call current_utc_time once. Then call format_timestamp with prefix \"UTC now: \" and \ that timestamp. After that, report the formatted result and stop.", ) .await?; ``` `current_utc_time` runs `date` through the tool executor of the agent’s context, which is local by default. `format_timestamp` takes a struct argument, and its doc comments become field descriptions in the tool’s JavaScript Object Notation (JSON) Schema. The loop counter behind `limit(6)` resets when the agent stops, so a second `query` gets six loops again. On the way to Bedrock, Swiftide turns the request into Converse messages: * Every tool is sent with automatic tool choice and strict schema validation. Set `tool_strict` to `false` to turn validation off. * Every system message becomes a Converse system block. * Adjacent tool outputs go back together in one user message. * Each `ChatMessage::Reasoning` in the message history becomes its own assistant message, sent before the assistant text or tool use it belongs to. Reasoning without both text and a signature, such as redacted reasoning, is dropped. On the way back, the client returns Claude’s reasoning in the `reasoning` field of `ChatCompletionResponse`. The agent then adds it to the message history as `ChatMessage::Reasoning`. ## Options Set defaults with `AwsBedrockBuilder::default_options`, or change them later with `options_mut`. [AWS Bedrock settings](/models/overview/#aws-bedrock-settings) lists the fields, the merge rules and the `reasoning_effort` beta header. A few details only matter here: | Field | Effect | | ------------------------------------------------------ | -------------------------------------------------------------- | | `prompt_model` | model ID or ARN; `default_prompt_model` sets it | | `max_tokens`, `temperature`, `top_p`, `stop_sequences` | unset fields are not sent | | `additional_model_response_field_paths` | JSON Pointer paths of model-specific response fields to return | `additional_model_request_fields` takes an `aws_smithy_types::Document`. Neither Swiftide nor the SDK re-exports that type, so add `aws-smithy-types` (1.x) to your crate to set it. `ReasoningEffort::Max` is meant for Claude Opus 4.6 with adaptive thinking. Set `thinking.type` to `adaptive` through `additional_model_request_fields` to use it. ### Reasoning and beta headers The swiftide examples do not set these fields, so this section names them without code. Build the value with `Options::builder()` and pass the result of `build()` to `default_options`. The setters take the value itself, not an `Option`: * `reasoning_effort` takes `swiftide::integrations::aws_bedrock_v2::ReasoningEffort` (`Low`, `Medium`, `High` or `Max`). Swiftide writes it to `output_config.effort` in the model request fields. * `tool_strict` takes a `bool`. `false` sends tools without strict schema validation. * `anthropic_beta` takes a list of beta header names. Swiftide adds them to the `anthropic_beta` array in the model request fields, without duplicates. * `additional_model_request_fields` takes the `Document` that the two fields above are merged into. See [`OptionsBuilder` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.OptionsBuilder.html) for every setter. ## Structured prompts `AwsBedrock` implements `DynStructuredPrompt`, so `structured_prompt::` from `StructuredPrompt` works through the blanket implementation. Swiftide sends the schema as a Converse output configuration with a JSON Schema text format, then parses the reply into JSON. Whether a model accepts that format is up to Bedrock. [Structured output](/models/structured-output/) shows the call. ## Errors and usage Throttling, unavailable services, models that are not ready, model timeouts, internal server errors, network failures, timeouts and unreadable responses are transient. For `complete_stream`, a model stream error is transient too. An error in the middle of a stream is transient only for throttling, unavailable services, internal server errors and model stream errors. Everything else is permanent. A response that stops because the context window is full, with no text, tool calls or reasoning, returns `ContextLengthExceeded`. Wrap the client in `LanguageModelWithBackOff` to retry transient errors; see [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/). The wrapper implements `SimplePrompt` and `ChatCompletion` but not `DynStructuredPrompt`, so a wrapped client loses `structured_prompt`. `on_usage` and `on_usage_async` receive the token `Usage` of each call. Its `cached_tokens` holds the cache-read input tokens. When there are none, it holds the cache-write tokens instead, so a first call that writes the prompt cache also reports cached input. Both callbacks run before the call returns; an `Err` from either fails the call. ## Reference * [`AwsBedrock`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrock.html) * [`AwsBedrockBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrockBuilder.html) * [`Options` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.Options.html) * [`ReasoningEffort` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/enum.ReasoningEffort.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`DynStructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.DynStructuredPrompt.html) * [`StructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.StructuredPrompt.html) * [`MetadataSummary`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataSummary.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`tool`](https://docs.rs/swiftide/0.34/swiftide/attr.tool.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`LanguageModelError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.LanguageModelError.html) ## Next [FastEmbed](/integrations/fastembed/) # FastEmbed > Embed chunks and queries with dense and sparse FastEmbed models, and rerank retrieved documents, on your own CPU without an API key. You embed chunks and queries with local dense and sparse models, pick a model that fits your data, and rerank retrieved documents, all on your CPU without an API key. Turn on the `fastembed` feature flag: ```sh cargo add swiftide@0.34 --features fastembed ``` ## Default models The hybrid search example builds one dense and one sparse embedder with the default models: examples/hybrid_search.rs ```rust // Ensure all batching is consistent let batch_size = 64; let fastembed_sparse = FastEmbed::try_default_sparse().unwrap().to_owned(); let fastembed = FastEmbed::try_default().unwrap().to_owned(); // Set up openai with the mini model, which is great for indexing let openai = openai::OpenAI::builder() .default_prompt_model("gpt-4o-mini") .build() .unwrap(); // Set up qdrant and use the combined fields (metadata + chunks) for both sparse and dense // vectors let qdrant = Qdrant::builder() .batch_size(batch_size) .vector_size(384) .with_vector(EmbeddedField::Combined) .with_sparse_vector(EmbeddedField::Combined) .collection_name("swiftide-hybrid-example") .build()?; ``` The excerpt also builds an OpenAI client and Qdrant storage. To compile it as is, add the `openai` and `qdrant` feature flags next to `fastembed`. `FastEmbed::try_default()` loads BGE (BAAI General Embedding) small en v1.5, a dense model with 384 dimensions. `FastEmbed::try_default_sparse()` loads SPLADE++ (Sparse Lexical and Expansion model, `prithivida/Splade_PP_en_v1`), a sparse model. Both return a `Result`, because loading the model can fail. `FastEmbed` implements `EmbeddingModel` and `SparseEmbeddingModel`. You pass the dense instance to the `Embed` batch transformer in the indexing pipeline and to `query_transformers::Embed` in the query pipeline. The sparse instance goes to `SparseEmbed` on both sides. Each instance holds one model. Passing a sparse instance to `Embed` fails with the permanent error “Expected dense model, got sparse”, and a dense instance in `SparseEmbed` fails with “Expected sparse model, got dense”. Your storage’s vector size must match the model: 384 for BGE small. ## Model choice The ask-your-codebase showcase indexes source code, so it picks a model trained on code: examples/ask-your-codebase/src/embedding.rs ```rust /// A code-aware embedding model that runs on the CPU and needs no API key. /// /// The model (~600MB) is downloaded on first use. pub fn local_model() -> Result { let model = TextEmbedding::try_new( InitOptions::new(EmbeddingModel::JinaEmbeddingsV2BaseCode).with_max_length(MAX_TOKENS), )?; FastEmbed::builder() .embedding_model(model) .batch_size(BATCH_SIZE) .build() } ``` `jina-embeddings-v2-base-code` produces 768-dimensional vectors. `fastembed::TextInitOptions::with_max_length` raises the token limit to 2048 for each embedded text. In ask-your-codebase that text is a chunk of up to 4096 bytes plus its generated questions and answers. FastEmbed truncates longer texts without an error. The default limit is 512 tokens, or less if the model’s own limit is lower. Swiftide does not re-export the `fastembed` crate. To name a model, add it yourself at the major version Swiftide uses: ```sh cargo add fastembed@5 --no-default-features ``` `FastEmbedBuilder::embedding_model` accepts a `fastembed::TextEmbedding` for dense vectors or a `fastembed::SparseTextEmbedding` for sparse ones. The fastembed documentation lists the dense models, with their dimensions, under `fastembed::EmbeddingModel`. The excerpt uses `fastembed::InitOptions`, which fastembed 5 keeps as a deprecated alias of `fastembed::TextInitOptions`. Use `fastembed::TextInitOptions` in new code to avoid the deprecation warning. Its options also set the cache directory (`fastembed::TextInitOptions::with_cache_dir`) and cap the ONNX (Open Neural Network Exchange) Runtime threads (`fastembed::TextInitOptions::with_intra_threads`), which default to every core. ## Batch size How the pipeline batch size and the storage batch size interact is covered in [Embedding](/indexing/embedding/#batch-sizes). The FastEmbed side has one cost to keep in mind: FastEmbed pads every text in a batch to the longest one, so long texts in a large batch cost memory. The docs-qa showcase sets 32 on both embedding steps and leaves FastEmbed at its default of 256. FastEmbed never sees more texts than the pipeline hands over, so its own batch size only matters when it is smaller: examples/docs-qa/src/index.rs ```rust indexing::Pipeline::from_loader(loader) .then_chunk(ChunkMarkdown::from_chunk_range( MIN_CHUNK_CHARS..MAX_CHUNK_CHARS, )) // A short title and keywords per chunk; both end up in the embedded text and help the // sparse (keyword) side of hybrid search in particular. .then( MetadataTitle::from_client(llm.clone()) .prompt_template(TITLE_PROMPT) .build()?, ) .then( MetadataKeywords::from_client(llm.clone()) .prompt_template(KEYWORDS_PROMPT) .build()?, ) .then_in_batch(Embed::new(embedders.dense.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_in_batch(SparseEmbed::new(embedders.sparse.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_store_with(qdrant.clone()) .run() .await ``` ask-your-codebase sets 32 on both the pipeline and the `FastEmbed` builder, because its embedded texts run up to 2048 tokens. To limit memory, lower the pipeline batch size first. ## Reranking `Rerank` is a response transformer that scores retrieved documents with a local cross-encoder and keeps the best ones. It implements `TransformResponse`. docs-qa builds it next to its answer generator: examples/docs-qa/src/query.rs ```rust let reranker = Rerank::builder() .top_k(CONTEXT_CHUNKS) .document_template(Some(RERANK_TEMPLATE.to_string())) .build()?; let answerer = answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(DOCUMENT_TEMPLATE) .build()?; ``` The default model is BGE reranker base. To use another one, pass a `fastembed::TextRerank` wrapped in `Arc>` to `Rerank::builder().model(...)`. `model_batch_size` sets the reranker’s batch size, which otherwise uses FastEmbed’s default of 256. For where `Rerank` sits in the query pipeline and how to write a rerank template, see [Reranking and response transformers](/querying/reranking-and-response-transformers/). ## First run and model cache The first time a model loads, FastEmbed downloads it from Hugging Face into `./.fastembed_cache` in the working directory. Later runs load it from there. Set `FASTEMBED_CACHE_DIR` to share one cache between projects. If `HF_HOME` is set, it takes precedence over both, and the models land there. The three docs-qa models (dense, sparse and reranker) take about 2.3 GB. The ask-your-codebase model takes about 600 MB. The ONNX Runtime library is downloaded once at build time, because the `fastembed` feature flag turns on fastembed’s `ort-download-binaries` feature. The first build therefore needs network access, and so does the first run. ## Gotchas * A `FastEmbed` keeps its model behind an `Arc>`. Clones share the same model, so calls on one instance and its clones run one at a time. A higher `with_concurrency` on `Embed` does not speed up embedding. * The model runs inside the async call, not on a blocking thread. A large batch occupies one Tokio worker thread until it finishes. * Indexing and querying must use the same model. See [Embedding](/indexing/embedding/) for what to do when you change it. * `Rerank::default()`, and `Rerank::builder().build()` without a model, panic when the default reranker cannot load, for example on a first run without network access. `FastEmbed::try_default()` returns an error instead. ## Reference * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`FastEmbedBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbedBuilder.html) * [`Rerank`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.Rerank.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`SparseEmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SparseEmbeddingModel.html) * [`TransformResponse`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.TransformResponse.html) * [`transformers::Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`transformers::SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.SparseEmbed.html) * [`query_transformers::Embed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.Embed.html) * [`query_transformers::SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.SparseEmbed.html) * [fastembed crate documentation](https://docs.rs/fastembed/5) * [Embedding](/indexing/embedding/) * [Reranking and response transformers](/querying/reranking-and-response-transformers/) * [Feature flags](/reference/feature-flags/) ## Next [Qdrant](/integrations/qdrant/) # LanceDB > Store nodes in an embedded LanceDB table in a local directory and retrieve them with a similarity search, a filter or your own LanceDB query. You end up with a vector store in a directory next to your code, with no database server to run. Swiftide persists nodes into a LanceDB table and retrieves them in a query pipeline. The integration sits behind the `lancedb` feature flag. The code on this page also uses `redb` for the node cache, `fastembed` for the embedder and `tree-sitter` for `MetadataQACode`: ```sh cargo add swiftide@0.34 --features lancedb,redb,fastembed,tree-sitter ``` ## Builder examples/ask-your-codebase/src/codebase.rs ```rust /// Remembers which files were fully indexed, so unchanged files are skipped next time. pub fn node_cache(&self) -> Result { std::fs::create_dir_all(&self.data_dir)?; Redb::builder() .database_path(self.data_dir.join("cache.redb")) .table_name("indexed_files") .build() } /// Embedded vector store with one vector per chunk and the source path next to it. pub fn vector_store(&self) -> Result { let uri = self.lancedb_dir(); LanceDB::builder() .uri(uri.to_str().context("Index path is not valid UTF-8")?) .table_name("code_chunks") .vector_size(embedding::DIMENSIONS) .with_vector(EmbeddedField::Combined) .with_metadata(PATH_FIELD) .build() } ``` The ask-your-codebase showcase keeps a `Redb` node cache and a `LanceDB` storage in the same directory. `uri` points at the LanceDB directory, as a Uniform Resource Identifier (URI). You need it unless you pass your own `connection_pool`. Without either, `build` returns the error `URI should be set`. `table_name` defaults to `swiftide`. `vector_size` is the size of every vector column and must match the embedder, here 768 for a FastEmbed code model. `with_vector` adds a vector column for an embedded field. `with_metadata` adds a column for a metadata field. The storage opens a pool of up to 10 connections, which `pool_size` changes. The table has these columns: | Column | Type | Content | | ----------------------- | -------------- | ----------------------------------------------------- | | `id` | 16 bytes | The node id, derived from the file path and the chunk | | `chunk` | text | The chunk | | one per `with_metadata` | text, nullable | The metadata value | | `vector_` | list of `f32` | The vector, for example `vector_combined` | Column names are lowercased, and every character that is not a letter or a digit becomes `_`. The field `MetadataQACode` writes, `Questions and Answers (code)`, becomes the column `questions_and_answers__code_`. ## Indexing pipeline examples/ask-your-codebase/src/index.rs ```rust let stats = indexing::Pipeline::from_loader( FileLoader::new(codebase.root()).with_extensions(&languages.extensions()), ) .filter(test_code::not_test_files(codebase.clone())) // Drop unit tests before the cache, which keys on path and content. Changing the content // after it would make the cache miss on every run. .then(test_code::without_unit_tests) .then(files_found.count()) // Skip files whose path and content are unchanged since the last successful run .filter_cached(codebase.node_cache()?) .then(files_changed.count()) .then(record_relative_path(codebase.clone())) // Split along syntax boundaries (functions, impls, ...) with tree-sitter .then_chunk(ChunkByLanguage::try_new(languages, CHUNK_SIZE)?) // Ask the LLM which questions each chunk answers; stored as metadata and embedded .then( MetadataQACode::from_client(llm) .prompt_template(QUESTIONS_PROMPT) .num_questions(QUESTIONS_PER_CHUNK) .concurrency(concurrent_requests) .build()?, ) // Embed chunk + metadata locally, in batches .then_in_batch(Embed::new(embedding::local_model()?).with_batch_size(embedding::BATCH_SIZE)) .then_store_with(codebase.vector_store()?) .run() .await?; ``` `then_store_with` hands the nodes to `LanceDB`, which implements the `Persist` trait. When the pipeline runs, it calls `setup` once. `setup` creates an empty table with the configured columns if the table does not exist, and opens the existing table otherwise. The storage writes 256 nodes per batch, which `batch_size` changes on the builder. Each batch is a merge insert on `id`. Ids come from the path and the chunk, so changed chunks add rows. See [Storage](/indexing/storage/) for how to clean them up. ## Metadata columns examples/ask-your-codebase/src/index.rs ```rust /// Stores the file path, relative to the codebase root, as metadata on the node. /// /// The vector store keeps this field next to each chunk, so answers can cite their sources. fn record_relative_path( codebase: Codebase, ) -> impl Fn(TextNode) -> Result + Clone + Send + Sync + 'static { move |mut node| { let path = codebase .relative_path(&node.path) .to_string_lossy() .into_owned(); node.metadata.insert(PATH_FIELD, path); Ok(node) } } ``` The closure transformer writes a `path` field into the node metadata, and `with_metadata(PATH_FIELD)` gives it a column. Only string values are stored. A number, a list or an object in that field is written as null, as is a field the node does not have. Metadata without a column is not stored at all. In the showcase, the questions from `MetadataQACode` are embedded with the chunk but have no column, so they never reach the table. ## Similarity search examples/ask-your-codebase/src/query.rs ```rust let search = SimilaritySingleEmbedding::default().with_top_k(top_k); let pipeline = query::Pipeline::from_search_strategy(search) // Rephrase the question into several, to widen what the search can match .then_transform_query( GenerateSubquestions::builder() .client(llm.clone()) .prompt_template(SUBQUESTIONS_PROMPT.into()) .build()?, ) // Embed with the same local model that embedded the chunks .then_transform_query(Embed::from_client(embedding::local_model()?)) .then_retrieve(codebase.vector_store()?) // Answer the original question from the retrieved code, each chunk under its file path .then_answer( answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(CHUNK_TEMPLATE) .build()?, ); let answered = pipeline.query(question).await?; ``` `LanceDB` implements the `Retrieve` trait for `SimilaritySingleEmbedding`. The search runs a nearest-neighbour query on the single vector column and returns `top_k` rows, 10 by default. `SimilaritySingleEmbedding::default()` is the search without a filter, and `with_top_k` chains onto it. The query needs an embedding, so an `Embed` query transformer runs before `then_retrieve`. Use the same model that embedded the chunks. A query without an embedding fails with `No embedding for query`. examples/ask-your-codebase/src/query.rs ```rust impl From> for Answer { /// Keeps the distinct file paths of the retrieved chunks, most relevant first. fn from(query: Query) -> Self { let mut sources: Vec = Vec::new(); for document in query.documents() { let Some(path) = document.metadata().get(PATH_FIELD).and_then(|v| v.as_str()) else { continue; }; if !sources.iter().any(|source| source == path) { sources.push(path.to_owned()); } } Self { text: query.answer().to_owned(), sources, } } } ``` Each retrieved document takes its content from the `chunk` column. Every other text column becomes a metadata entry, keyed by the column name. `path` keeps its name. Vector and id columns are left out. A metadata column that holds null for a row comes back as an empty string, not as a missing key. The showcase’s `ask` run, answered by `claude-sonnet-5-5`, listed these `path` values: ```text Sources: - src/agent.rs - src/tools/control.rs - src/lib.rs - src/state.rs - src/errors.rs - src/task_agent.rs - src/hooks.rs - src/tasks.rs - src/default_context.rs ``` ## Filters and custom queries `SimilaritySingleEmbedding::from_filter` takes a `String` for LanceDB. Swiftide passes it unchanged to LanceDB’s `only_if`, as a Structured Query Language (SQL) filter on the column names. For example, `SimilaritySingleEmbedding::from_filter("path = 'src/agent.rs'".to_string())` searches only the chunks of that file. For anything else, use `CustomStrategy` with a LanceDB query. `LanceDB` accepts any query that implements LanceDB’s `ExecutableQuery` trait, and turns the rows into documents as above. With `CustomStrategy::from_async_query`, the closure returns a future that awaits `get_connection` or `open_table` and builds the query. `CustomStrategy::from_query` takes a synchronous closure that cannot await, so open the `lancedb::Table` beforehand and capture it. Swiftide does not re-export the `lancedb` crate. Add it at the version Swiftide uses to name its types: ```sh cargo add lancedb@0.33 --no-default-features ``` Swiftide’s integration test [`swiftide/tests/lancedb.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide/tests/lancedb.rs) builds a custom vector search with `from_async_query`. ## Remote storage `api_key`, `region` and `storage_options` on the builder go to every LanceDB connection. Swiftide enables the `remote` feature of the `lancedb` crate, so a `db://` URI plus `api_key` and `region` connects to LanceDB Cloud. Object stores such as Amazon Simple Storage Service (S3) need a storage feature of the `lancedb` crate, for example `aws`, that Swiftide does not enable. Add `lancedb` with that feature to your own `Cargo.toml`, and Cargo combines it with Swiftide’s. ## Gotchas * A vector column without a size panics in `build` with `Vector size should be set either in the field or in the LanceDB builder`. Set `vector_size`. * `setup` never changes an existing table. After a change to the vector size, the columns or the embedding model, use a new table name or delete the directory. In the showcase, delete `.ask-your-codebase/` to start over. * `SimilaritySingleEmbedding` needs exactly one vector column. With zero or several, retrieval fails with `Zero or multiple vector fields configured in schema`. There is no `HybridSearch` support. * LanceDB creates no vector index for you. For large tables, create one through `get_connection`. * LanceDB’s dependencies compile Protocol Buffers definitions at build time, so the build needs `protoc` on your `PATH`. Install it with your package manager, for example `brew install protobuf` or `apt install protobuf-compiler`. ## Reference * [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html) * [`LanceDBBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDBBuilder.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [`Redb`](https://docs.rs/swiftide/0.34/swiftide/integrations/redb/struct.Redb.html) * [`Pipeline::filter_cached`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.filter_cached) * [`Pipeline::then_store_with`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.then_store_with) * [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html) * [`Retrieve`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Retrieve.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`CustomStrategy`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.CustomStrategy.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) ## Next [Tree-sitter](/integrations/tree-sitter/) # OpenAI and Azure > Build an OpenAI or Azure OpenAI client for prompts, chat completions, agents, embeddings and the Responses API, and point it at any OpenAI-compatible base URL. Here you build an `OpenAI` client, or the same client for an Azure OpenAI deployment, and pass it to indexing pipelines, query pipelines and agents. The integration sits behind the `openai` feature flag: ```sh cargo add swiftide@0.34 --features openai ``` ## Client ```rust let openai = swiftide::integrations::openai::OpenAI::builder() .default_prompt_model("gpt-4o-mini") .default_embed_model("text-embedding-3-small") .build()?; ``` `default_prompt_model` sets the model for `prompt`, `structured_prompt`, `complete` and `complete_stream`. `default_embed_model` sets the model for `embed`. The client has no default model. A call without one fails with a permanent error that says `Model not set`. Unless you call `api_base`, `api_key` or `client(..)`, the builder creates an `async_openai::Client` with `OpenAIConfig::default()`. That configuration reads `OPENAI_API_KEY` and `OPENAI_BASE_URL` from the environment, and `OPENAI_ORG_ID` and `OPENAI_PROJECT_ID` when they are set. The base URL defaults to `https://api.openai.com/v1`. `OpenAI` implements `SimplePrompt`, `ChatCompletion`, `EmbeddingModel` and `StructuredPrompt`. So one client serves a large language model (LLM) transformer, the `Embed` batch transformer, an answer generator and `Agent::builder().llm(&openai)`. Agents also need the `swiftide-agents` feature flag. Cloning the client is cheap: the clones share one HTTP client. ## Custom base URL examples/ask-your-codebase/src/llm.rs ```rust impl LlmEndpoint { /// Builds a client for this endpoint. /// /// Rate limits and transient errors are retried with exponential backoff, which matters when /// indexing fires many prompts concurrently. After `retry_budget` the error is returned, so /// an unreachable or broken endpoint fails instead of hanging. pub fn client(&self, retry_budget: Duration) -> Result> { let openai = OpenAI::builder() .api_base(&self.base_url) .api_key(&self.api_key) .default_prompt_model(&self.model) .use_responses_api(matches!(self.api, OpenAiApi::Responses)) .build()?; let backoff = BackoffConfiguration { max_elapsed_time_sec: retry_budget.as_secs(), ..BackoffConfiguration::default() }; Ok(Arc::new(LanguageModelWithBackOff::new(openai, backoff))) } } ``` The ask-your-codebase showcase sends its prompts to any OpenAI-compatible endpoint. `LlmEndpoint` holds the base URL, key and model from the command line. `api_base` and `api_key` on the builder override the base URL and key from the environment. Everything else still comes from `OpenAIConfig::default()`, so `OPENAI_ORG_ID` and `OPENAI_PROJECT_ID` are sent as headers when they are set, also to a third-party gateway. The showcase picks the Responses API or Chat Completions from a command-line flag. The showcase wraps the client in `LanguageModelWithBackOff`. Transient errors, such as rate limits and outages, are retried with exponential backoff until `max_elapsed_time_sec` runs out. Permanent errors, including quota and billing errors that arrive as HTTP 429, return at once. `LlmEndpoint::client` returns `Arc`, so the rest of the showcase is provider-independent. [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/) covers the settings. For anything the two setters do not cover, such as a custom HTTP client or another configuration type, pass a whole `async_openai::Client` to `client(..)`. It replaces what `api_base` and `api_key` set, and whichever you call last wins. Swiftide re-exports `OpenAIConfig`, `AzureConfig` and `ReasoningEffort` from `async-openai` at `swiftide::integrations::openai`. It does not re-export `async_openai::Client`, so add the crate at the version Swiftide uses: ```sh cargo add async-openai@0.41 --no-default-features ``` ## Azure OpenAI Azure OpenAI uses its own URL scheme and authentication header, so it needs `AzureConfig` instead of `OpenAIConfig`: ```rust let config = AzureConfig::new() .with_api_base("https://my-resource.openai.azure.com") .with_api_version("2024-10-21") .with_deployment_id("gpt-4o-mini") .with_api_key(std::env::var("AZURE_OPENAI_API_KEY")?); let azure: GenericOpenAI = GenericOpenAIBuilder::default() .client(async_openai::Client::with_config(config)) .default_prompt_model("gpt-4o-mini") .build()?; let answer = azure.prompt("Name one Rust web framework.".into()).await?; ``` `OpenAI` is a type alias for `GenericOpenAI`. For Azure you build a `GenericOpenAI` through `GenericOpenAIBuilder` and pass the client with `client(..)`. `api_base` and `api_key` exist only on the `OpenAI` builder. The Azure client implements the same model traits. `AzureConfig::with_api_base` takes the resource URL without a path. Requests go to `{api_base}/openai/deployments/{deployment_id}/...`, with `api-version` as a query parameter and the key in an `api-key` header. Without `with_api_key`, `AzureConfig` falls back to `OPENAI_API_KEY`. A deployment serves one model, so build one client per deployment, for example one for chat and one for embeddings. Set `default_prompt_model` or `default_embed_model` anyway: Swiftide refuses to send a request without a model, even though Azure routes by deployment. `GenericOpenAI` is a different type from `OpenAI`. A field typed `OpenAI` does not accept it. Store the client as `Arc` or another trait object when the provider varies. See [Clients as trait objects](/models/overview/#clients-as-trait-objects). ## Embeddings ```rust let openai = OpenAI::builder() .default_embed_model("text-embedding-3-small") .default_options(Options::builder().dimensions(512)) .build()?; let vectors = openai .embed(vec!["What is a borrow checker?".into()]) .await?; println!("{} dimensions", vectors[0].len()); indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) .then_in_batch(Embed::new(openai.clone()).with_batch_size(64)) .then_store_with(MemoryStorage::default()) .run() .await?; ``` `embed` sends a list of texts in one request and returns one vector per text. Embedding requests carry two settings from `Options`: `dimensions` and `user`. `dimensions` works with text-embedding-3 and later models. The vector size you get must match the vector size of your storage. In an indexing pipeline, the `Embed` batch transformer calls `embed` once per batch. `with_batch_size(64)` sets the batch size. Without it, the pipeline default of 256 applies. Use the same model and dimensions for the `Embed` query transformer, so queries and chunks land in the same vector space. ## Request options [Request settings](/models/overview/#request-settings) covers `default_options` and which call sends which field. Two details are specific to this client. `parallel_tool_calls` is unset by default, although the builder method’s documentation says it defaults to `true`. Once you set it, the client sends it on every chat request, including `prompt`, `structured_prompt` and `complete` calls without tools. OpenAI rejects `parallel_tool_calls` on a request without tools. Give your agents a separate client when you set it, and keep it off the client that serves transformers and answer generators. `extra_body` merges extra fields into the top level of the request body. Only `complete` and `complete_stream` send it, on the Chat Completions API and the Responses API alike. `prompt`, `structured_prompt` and `embed` do not. ## Responses API ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-4.1-mini") .default_options(Options::builder().temperature(0.2)) .use_responses_api(true) .build()?; ``` `use_responses_api(true)` sends `complete`, `complete_stream`, `prompt` and `structured_prompt` to the OpenAI Responses API instead of the Chat Completions API. `embed` is unaffected. The flag is off by default. With `reasoning_effort` set, the client asks for reasoning summaries and encrypted reasoning content, and sends the encrypted content back on the next turn. Summaries are not sent back. `reasoning_features(false)` turns that off. The Responses API ignores `seed` and `presence_penalty` and logs a warning. [Responses API and reasoning](/models/overview/#responses-api-and-reasoning) shows an agent that keeps its reasoning in the message history. ## Streaming and usage `stream_full(false)` makes `complete_stream` send only the delta in every item but the last. See [Deltas only](/models/overview/#deltas-only). `on_usage` and `on_usage_async` report the token `Usage` of every request, embeddings included. The `on_usage_async` callback returns a pinned, boxed future. See [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/). ## Gotchas * Some OpenAI-compatible gateways cap unset `max_completion_tokens` at 4096 and truncate silently; see [OpenAI-compatible providers](/integrations/openai-compatible/#gotchas). * A gateway’s Responses API has to match OpenAI’s response schema exactly. The captured docs-qa run uses Chat Completions, its default, because its gateway returns `"input_tokens_details": null`, which `async-openai` rejects. ## Reference * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`OpenAIBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAIBuilder.html) * [`GenericOpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html) * [`GenericOpenAIBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html) * [`OpenAIConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.OpenAIConfig.html) * [`AzureConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.AzureConfig.html) * [`Options`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.Options.html) * [`OptionsBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.OptionsBuilder.html) * [`ReasoningEffort`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/enum.ReasoningEffort.html) * [`GenericOpenAIBuilder::use_responses_api`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.use_responses_api) * [`GenericOpenAIBuilder::stream_full`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.stream_full) * [`GenericOpenAIBuilder::on_usage`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.on_usage) * [`GenericOpenAIBuilder::parallel_tool_calls`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.parallel_tool_calls) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`BackoffConfiguration`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.BackoffConfiguration.html) * [`Usage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.Usage.html) ## Next [OpenAI-compatible providers](/integrations/openai-compatible/) # OpenAI-compatible providers > Build a client for Ollama, Groq, Gemini, OpenRouter, Dashscope or any other server that speaks the OpenAI API, with the feature flag, base URL and key variable each one uses. Swiftide talks to Ollama, Groq, Gemini, OpenRouter and Dashscope through the same OpenAI client, each with its own base Uniform Resource Locator (URL) and key. You end up with a working client for each, local embeddings through Ollama, and a client for any other gateway that speaks the OpenAI application programming interface (API). | Client | Feature flag | Base URL | Key variable | | ------------ | ------------- | --------------------------------------------------------- | ---------------------------------------- | | `Ollama` | `ollama` | `http://localhost:11434/v1` | none | | `Groq` | `groq` | `https://api.groq.com/openai/v1` | `GROQ_API_KEY` | | `Gemini` | `gemini` | `https://generativelanguage.googleapis.com/v1beta/openai` | `GEMINI_API_KEY` | | `OpenRouter` | `open-router` | `https://openrouter.ai/api/v1` | `OPENROUTER_API_KEY` | | `Dashscope` | `dashscope` | `https://dashscope.aliyuncs.com/compatible-mode/v1` | `QWEN_API_KEY`, then `DASHSCOPE_API_KEY` | Each of these feature flags also turns on `openai`: ```sh cargo add swiftide@0.34 --features ollama ``` ## Ollama examples/index_ollama.rs ```rust let ollama_client = integrations::ollama::Ollama::default() .with_default_prompt_model("llama3.1") .to_owned(); let fastembed = integrations::fastembed::FastEmbed::try_default()?; let memory_store = MemoryStorage::default(); ``` Every client on this page is `GenericOpenAI` with a provider-specific configuration type. It shares the `Options` and the model traits of [`OpenAI`](/integrations/openai/). `Ollama::default()` builds a client without a model. `with_default_prompt_model` takes `&mut self` and returns `&mut Self`, so `.to_owned()` turns the result back into a client you own. The builder, `Ollama::builder()`, avoids that step. The example uses Ollama only for the `MetadataQAText` transformer and embeds the chunks with FastEmbed, so it also needs the `fastembed` feature flag. ### Local embeddings Ollama also serves embeddings. Set an embed model and use the same client on both sides: ```rust let ollama = Ollama::builder() .default_prompt_model("llama3.2") .default_embed_model("nomic-embed-text") .build()?; // nomic-embed-text returns 768 dimensions. let qdrant = Qdrant::builder() .vector_size(768) .collection_name("local-docs") .build()?; indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) .then_in_batch(Embed::new(ollama.clone()).with_batch_size(32)) .then_store_with(qdrant.clone()) .run() .await?; let answered = query::Pipeline::default() .then_transform_query(query_transformers::Embed::from_client(ollama.clone())) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(ollama)) .query("How do I configure the loader?") .await?; ``` The `Embed` batch transformer embeds the chunks and the `Embed` query transformer embeds the question. Both must use the same model, or the vectors do not match. The storage vector size must match the model’s output too. nomic-embed-text returns 768 dimensions. The snippet stores the vectors in Qdrant, so it also needs the `qdrant` feature flag and a running Qdrant server. [Qdrant](/integrations/qdrant/) covers that part. Pull the models before you run it: ```sh ollama pull nomic-embed-text ollama pull llama3.2 ``` The Ollama example above uses `llama3.1`. Pull that one to run it. ### Remote Ollama `OllamaConfig` is public, so you can point the client at another machine: ```rust let mut config = OllamaConfig::default(); config.with_api_base("http://gpu-box:11434/v1"); let ollama = Ollama::builder() .client(async_openai::Client::with_config(config)) .default_prompt_model("llama3.2") .build()?; let answer = ollama.prompt("Name one Rust web framework.".into()).await?; ``` `OllamaConfig::with_api_base` takes `&mut self`, so build the config in a mutable variable first. `async_openai::Client` is not re-exported. Add the crate at the version Swiftide uses: ```sh cargo add async-openai@0.41 --no-default-features ``` `OllamaConfig` sends no `Authorization` header and reads no environment variable. For an Ollama server behind a proxy that checks a bearer token, use `OpenAI` with `api_base` and `api_key` instead. See [Other gateways](#other-gateways). ## Groq examples/index_groq.rs ```rust let groq_client = integrations::groq::Groq::builder() .default_prompt_model("llama3-8b-8192") .to_owned() .build()?; let fastembed = integrations::fastembed::FastEmbed::try_default()?; let memory_store = MemoryStorage::default(); ``` `Groq::builder()` gives a builder with the Groq base URL and the key from `GROQ_API_KEY`. The `.to_owned()` in the middle clones the builder and changes nothing. Like the Ollama example, this one embeds with FastEmbed and needs the `fastembed` feature flag. Groq has retired `llama3-8b-8192`, the model in the example. Pick a current model from Groq’s model list. ## Gemini examples/hello_agents.rs ```rust let openai = swiftide::integrations::gemini::Gemini::builder() .default_embed_model("gemini-embedding-exp-03-07") .default_prompt_model("gemini-2.0-flash") .build()?; ``` The variable is named `openai`, but its type is `Gemini`. Later in the example it goes to `Agent::builder().llm(&openai)`, which works because `Gemini` implements `ChatCompletion`. The agent needs the `swiftide-agents` feature flag as well. The key comes from `GEMINI_API_KEY`. The embed model in the example, `gemini-embedding-exp-03-07`, is experimental. Check Google’s model list for a current embedding model before you copy it. ## OpenRouter ```rust let openrouter = OpenRouter::builder() .default_prompt_model("anthropic/claude-sonnet-4.5") .build()?; let config = OpenRouterConfig::builder() .api_key(std::env::var("TEAM_OPENROUTER_KEY")?) .site_url("https://example.com/my-app") .site_name("My App") .build()?; let team_openrouter = OpenRouter::builder() .client(async_openai::Client::with_config(config)) .default_prompt_model("openai/gpt-5-mini") .build()?; ``` The first client reads `OPENROUTER_API_KEY`. The second takes the key from an `OpenRouterConfig`, built with `OpenRouterConfig::builder()`. That builder requires `api_key`, `site_url` and `site_name`. Leave one out and `build()` returns an error at runtime. `site_url` sets the `HTTP-Referer` header, which OpenRouter uses for its app rankings. At this release the `X-Title` header also takes the `site_url` value, so `site_name` is required but not sent. `OpenRouterConfig::default()` leaves both unset. It then sends `https://github.com/bosun-ai/swiftide` as `HTTP-Referer` and `Swiftide` as `X-Title`. ## Dashscope examples/dashscope.rs ```rust let client = DashscopeBuilder::default() .default_embed_model("text-embedding-v2") .default_prompt_model("qwen-long") .build()?; let tempdir = TempDir::new().unwrap(); let lancedb = LanceDB::builder() .uri(tempdir.child("lancedb").to_str().unwrap()) .vector_size(1536) .with_vector(EmbeddedField::Combined) .with_metadata(metadata_qa_text::NAME) .table_name("swiftide_test") .build() .unwrap(); ``` `DashscopeBuilder::default()` is the same as `Dashscope::builder()`. The client prompts with `qwen-long` and embeds with `text-embedding-v2`. The example stores the vectors in LanceDB with a vector size of 1536, so it also needs the `lancedb` feature flag. [LanceDB](/integrations/lancedb/) covers that part. ## Other gateways The configuration types of `Groq`, `Gemini` and `Dashscope` are private. To use another key variable or another base URL with them, use `OpenAI` with `api_base` and `api_key`. ```rust let groq = OpenAI::builder() .api_base("https://api.groq.com/openai/v1") .api_key(std::env::var("TEAM_GROQ_KEY")?) .default_prompt_model("llama-3.3-70b-versatile") .build()?; ``` For vLLM, LiteLLM or a company gateway, see [Custom base URL](/integrations/openai/#custom-base-url). ## Gotchas * None of the clients has a default model. Set one with `default_prompt_model` or `default_embed_model`, or calls fail with `Model not set`. * A client reads its key when you build it. For `Groq`, `Gemini` and `Dashscope`, a missing key becomes an empty one, and the provider rejects the first request. `OpenRouter` panics on the first request instead. * Each alias is its own type. To pick the provider at runtime, see [Clients as trait objects](/models/overview/#clients-as-trait-objects). * The endpoint decides what works. Every client implements `EmbeddingModel` and `StructuredPrompt`, but a provider without embeddings or structured output returns an error at runtime. * Every client uses the Chat Completions API by default. `use_responses_api(true)` exists on every builder, but only some providers serve the Responses API. * For request fields that only one provider knows, set `extra_body` in `Options`. Only `complete` and `complete_stream` send it. `prompt`, `structured_prompt` and `embed` do not. See [Extra request fields](/models/overview/#extra-request-fields). * Some gateways cap the output at 4096 tokens when `max_completion_tokens` is unset. Set it in `Options`, because a cut-off answer looks complete. ## Reference * [`GenericOpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html) * [`GenericOpenAI::with_default_prompt_model`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html#method.with_default_prompt_model) * [`GenericOpenAIBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html) * [`Options`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.Options.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`Ollama`](https://docs.rs/swiftide/0.34/swiftide/integrations/ollama/type.Ollama.html) * [`OllamaConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/ollama/config/struct.OllamaConfig.html) * [`OllamaConfig::with_api_base`](https://docs.rs/swiftide/0.34/swiftide/integrations/ollama/config/struct.OllamaConfig.html#method.with_api_base) * [`Groq`](https://docs.rs/swiftide/0.34/swiftide/integrations/groq/type.Groq.html) * [`Gemini`](https://docs.rs/swiftide/0.34/swiftide/integrations/gemini/type.Gemini.html) * [`OpenRouter`](https://docs.rs/swiftide/0.34/swiftide/integrations/open_router/type.OpenRouter.html) * [`OpenRouterConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/open_router/config/struct.OpenRouterConfig.html) * [`OpenRouterConfigBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/open_router/config/struct.OpenRouterConfigBuilder.html) * [`Dashscope`](https://docs.rs/swiftide/0.34/swiftide/integrations/dashscope/type.Dashscope.html) * [`DashscopeBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/dashscope/type.DashscopeBuilder.html) * [`MetadataQAText`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQAText.html) * [`Embed` (indexing)](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`Embed` (query)](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.Embed.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`StructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.StructuredPrompt.html) ## Next [Anthropic](/integrations/anthropic/) # Integrations > Lists every Swiftide integration with the feature flag that enables it, the traits it implements, its main types, examples and guide. Each Swiftide integration sits behind its own feature flag. The matrix below shows which flag to add, what the integration plugs into, and where its guide is. An integration is a model provider, storage backend, node cache, loader, transformer, agent tool, token estimator or observability layer that lives behind a feature flag of the `swiftide` crate. ## Feature flags Name the feature flags of the integrations you use when you add `swiftide`: ```sh cargo add swiftide@0.34 --features qdrant,fastembed ``` Feature flags are additive. Each one compiles its integration and the flags it depends on. Without its flag, an integration’s module does not exist: `swiftide::integrations::qdrant` needs `qdrant`. Only `macros` is on by default. Most integrations live under `swiftide::integrations`, which re-exports the `swiftide-integrations` crate with the same feature names. A few sit elsewhere. MCP (Model Context Protocol) tools are in `swiftide::agents::tools::mcp`, Langfuse is under `swiftide::langfuse`, and the tree-sitter transformers are also re-exported under `swiftide::indexing::transformers`. Some flags turn on other flags: * `groq`, `gemini`, `ollama`, `open-router` and `dashscope` each turn on `openai`. Their clients build on the OpenAI client. * `mcp` and `langfuse` each turn on `swiftide-agents`, which turns on `swiftide-tasks`. You do not need to add `swiftide-agents` next to them. Some flags compile native code or large SDKs (software development kits) and make builds slower: * `fastembed` downloads ONNX (Open Neural Network Exchange) Runtime binaries at build time. * `lancedb` compiles an embedded database. It also needs a `protoc` binary on your `PATH` at build time. * `duckdb` links a DuckDB library that is already on your system. Point `DUCKDB_LIB_DIR` at it, make it findable through pkg-config, or set `DUCKDB_DOWNLOAD_LIB=1` to download it during the build. * `kafka` builds librdkafka from source with CMake. * `fluvio` compiles the Fluvio client. * `aws-bedrock` compiles the AWS (Amazon Web Services) SDK. Turn these on only in the crates that need them. ## The all feature The `all` flag turns on ten common integrations, but not `anthropic`, `lancedb`, `mcp` or `swiftide-agents`. It also includes `fastembed` and `aws-bedrock`, so it builds slowly. [The all bundle](/reference/feature-flags/#the-all-bundle) lists exactly what it includes. ## TLS The HTTP (Hypertext Transfer Protocol) clients use rustls for TLS (Transport Layer Security) and need no setup: each uses the rustls crypto provider your application installed, or else its own. The exception is Redis over `rediss://`, where Swiftide installs aws-lc-rs as the process-wide provider before connecting if none is installed, so install yours before the first Redis call to use another. The `fastembed` model download goes through native TLS, which means OpenSSL on Linux. ## Matrix columns Every flag, type and example in the matrix matches Swiftide 0.34. * **Integration** names the provider or backend, with a short note. * **Feature flag** is the flag to pass to `cargo add`. * **Provides** says where the integration plugs in: * **LLM**: a large language model (LLM) client for prompts, chat completions and agents (`SimplePrompt`, `ChatCompletion`). * **Embeddings** and **Sparse embeddings**: an embedder for dense or sparse vectors (`EmbeddingModel`, `SparseEmbeddingModel`). * **Structured output**: typed responses through `StructuredPrompt`. * **Storage**: a backend that the indexing pipeline stores nodes in (`Persist`). * **Retrieval**: a retriever for the query pipeline (`Retrieve`). The storage guide for each backend lists the search strategies it supports. * **Node cache**: a node cache that skips nodes an indexing pipeline has already processed (`NodeCache`). * **Loader**: a loader that produces nodes (`Loader`). * **Transformers**: transformers and chunkers for the indexing pipeline. * **Reranking**: a response transformer that reorders retrieved documents (`TransformResponse`). * **Agent tooling**: tools for agents, or token estimation. * **Observability**: traces or metrics. * **Main types** are the types you build. Each links to docs.rs for the pinned release. * **Examples** link to example files in the Swiftide repository at the pinned commit. * **Guide** is the page on this site that shows the integration in use. | Integration | Feature flag | Provides | Main types | Examples | Guide | | -------------------------------------------------------------------------------------------------------- | ------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ | | OpenAI: Chat Completions and the Responses API; Azure OpenAI through `GenericOpenAI` with `AzureConfig`. | `openai` | LLM, Embeddings, Structured output | [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html), [`GenericOpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html) | [`responses_api.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api.rs), [`responses_api_reasoning.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api_reasoning.rs), [`structured_prompt.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/structured_prompt.rs), [`describe_image.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/describe_image.rs) | [integrations/openai](/integrations/openai/) | | Anthropic | `anthropic` | LLM | [`Anthropic`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.Anthropic.html) | [`streaming_agents.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/streaming_agents.rs) | [integrations/anthropic](/integrations/anthropic/) | | AWS Bedrock: Uses the Bedrock Converse API. | `aws-bedrock` | LLM, Structured output | [`AwsBedrock`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrock.html) | [`aws_bedrock.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock.rs), [`aws_bedrock_agent.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock_agent.rs) | [integrations/aws-bedrock](/integrations/aws-bedrock/) | | Groq: OpenAI-compatible client preconfigured for Groq. | `groq` | LLM, Structured output | [`Groq`](https://docs.rs/swiftide/0.34/swiftide/integrations/groq/type.Groq.html) | [`index_groq.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_groq.rs) | [integrations/openai-compatible](/integrations/openai-compatible/) | | Google Gemini: OpenAI-compatible client preconfigured for the Gemini API. | `gemini` | LLM, Embeddings, Structured output | [`Gemini`](https://docs.rs/swiftide/0.34/swiftide/integrations/gemini/type.Gemini.html) | [`hello_agents.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/hello_agents.rs) | [integrations/openai-compatible](/integrations/openai-compatible/) | | Ollama: OpenAI-compatible client preconfigured for a local Ollama server. | `ollama` | LLM, Embeddings, Structured output | [`Ollama`](https://docs.rs/swiftide/0.34/swiftide/integrations/ollama/type.Ollama.html) | [`index_ollama.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_ollama.rs), [`index_md_into_pgvector.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_md_into_pgvector.rs) | [integrations/openai-compatible](/integrations/openai-compatible/) | | OpenRouter: OpenAI-compatible client preconfigured for OpenRouter. | `open-router` | LLM, Structured output | [`OpenRouter`](https://docs.rs/swiftide/0.34/swiftide/integrations/open_router/type.OpenRouter.html) | - | [integrations/openai-compatible](/integrations/openai-compatible/) | | Dashscope: OpenAI-compatible client preconfigured for Alibaba Dashscope. | `dashscope` | LLM, Embeddings, Structured output | [`Dashscope`](https://docs.rs/swiftide/0.34/swiftide/integrations/dashscope/type.Dashscope.html) | [`dashscope.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/dashscope.rs) | [integrations/openai-compatible](/integrations/openai-compatible/) | | FastEmbed: Runs dense, sparse and reranking models locally with ONNX Runtime. | `fastembed` | Embeddings, Sparse embeddings, Reranking | [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html), [`Rerank`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.Rerank.html) | [`fastembed.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/fastembed.rs), [`hybrid_search.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/hybrid_search.rs), [`reranking.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/reranking.rs) | [integrations/fastembed](/integrations/fastembed/) | | Qdrant: Similarity and hybrid (dense + sparse) search, named vectors, filters. | `qdrant` | Storage, Retrieval | [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) | [`index_codebase.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_codebase.rs), [`hybrid_search.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/hybrid_search.rs), [`store_multiple_vectors.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/store_multiple_vectors.rs) | [integrations/qdrant](/integrations/qdrant/) | | PgVector | `pgvector` | Storage, Retrieval | [`PgVector`](https://docs.rs/swiftide/0.34/swiftide/integrations/pgvector/struct.PgVector.html) | [`index_md_into_pgvector.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_md_into_pgvector.rs) | [integrations/pgvector](/integrations/pgvector/) | | LanceDB: Embedded, no server required; also supports custom queries. | `lancedb` | Storage, Retrieval | [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html) | [`lancedb.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/lancedb.rs), [`dashscope.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/dashscope.rs) | [integrations/lancedb](/integrations/lancedb/) | | DuckDB: Embedded; similarity and hybrid search. | `duckdb` | Storage, Retrieval, Node cache | [`Duckdb`](https://docs.rs/swiftide/0.34/swiftide/integrations/duckdb/struct.Duckdb.html) | - | [indexing/storage](/indexing/storage/) | | Redis: Also stores agent message history. | `redis` | Storage, Node cache | [`Redis`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.Redis.html) | [`index_into_redis.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_into_redis.rs), [`index_codebase.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_codebase.rs) | [indexing/caching](/indexing/caching/) | | Redb: Embedded node cache, no server required. | `redb` | Node cache | [`Redb`](https://docs.rs/swiftide/0.34/swiftide/integrations/redb/struct.Redb.html) | - | [indexing/caching](/indexing/caching/) | | Kafka | `kafka` | Loader, Storage | [`Kafka`](https://docs.rs/swiftide/0.34/swiftide/integrations/kafka/struct.Kafka.html) | [`kafka.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/kafka.rs) | [indexing/loading](/indexing/loading/) | | Fluvio | `fluvio` | Loader | [`Fluvio`](https://docs.rs/swiftide/0.34/swiftide/integrations/fluvio/struct.Fluvio.html) | [`fluvio.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/fluvio.rs) | [indexing/loading](/indexing/loading/) | | Parquet | `parquet` | Loader | [`Parquet`](https://docs.rs/swiftide/0.34/swiftide/integrations/parquet/struct.Parquet.html) | - | [indexing/loading](/indexing/loading/) | | Scraping (spider) | `scraping` | Loader, Transformers | [`ScrapingLoader`](https://docs.rs/swiftide/0.34/swiftide/integrations/scraping/struct.ScrapingLoader.html), [`HtmlToMarkdownTransformer`](https://docs.rs/swiftide/0.34/swiftide/integrations/scraping/struct.HtmlToMarkdownTransformer.html) | [`scraping_index_to_markdown.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/scraping_index_to_markdown.rs) | [indexing/loading](/indexing/loading/) | | Tree-sitter: Code-aware chunking, outlines and metadata; Perl behind `tree-sitter-perl`. | `tree-sitter` | Transformers | [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkCode.html), [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQACode.html), [`OutlineCodeTreeSitter`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.OutlineCodeTreeSitter.html), [`MetadataRefsDefsCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataRefsDefsCode.html) | [`index_codebase.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_codebase.rs), [`index_codebase_reduced_context.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_codebase_reduced_context.rs) | [integrations/tree-sitter](/integrations/tree-sitter/) | | MCP: Exposes the tools of any MCP server (via rmcp) to an agent. | `mcp` | Agent tooling | [`McpToolbox`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/mcp/struct.McpToolbox.html) | [`agents_mcp_tools.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agents_mcp_tools.rs) | [agents/mcp](/agents/mcp/) | | tiktoken: Token estimation with OpenAI tokenizers. | `tiktoken` | Agent tooling | [`TikToken`](https://docs.rs/swiftide/0.34/swiftide/integrations/tiktoken/struct.TikToken.html) | - | [models/errors-backoff-and-usage](/models/errors-backoff-and-usage/) | | Langfuse: A tracing layer that sends agent and pipeline spans to Langfuse. | `langfuse` | Observability | [`LangfuseLayer`](https://docs.rs/swiftide/0.34/swiftide/langfuse/struct.LangfuseLayer.html) | [`langfuse.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/langfuse.rs) | [observability/langfuse](/observability/langfuse/) | | Metrics: Emits usage and pipeline metrics through the `metrics` crate facade. | `metrics` | Observability | - | [`usage_metrics.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/usage_metrics.rs) | [observability/metrics-and-statistics](/observability/metrics-and-statistics/) | ## Guides This section has a page for each of these integrations: * [OpenAI and Azure](/integrations/openai/): Chat Completions, the Responses API (application programming interface), and Azure OpenAI. * [OpenAI-compatible providers](/integrations/openai-compatible/): Groq, Gemini, Ollama, OpenRouter, Dashscope and any OpenAI-compatible gateway. * [Anthropic](/integrations/anthropic/): prompts, chat completions, streaming and agents with Claude. * [AWS Bedrock](/integrations/aws-bedrock/): Bedrock models through the Converse API. * [FastEmbed](/integrations/fastembed/): dense and sparse embeddings and reranking on your CPU (central processing unit). * [Qdrant](/integrations/qdrant/): storage with similarity and hybrid search. * [PgVector](/integrations/pgvector/): storage in Postgres with the pgvector extension. * [LanceDB](/integrations/lancedb/): an embedded vector store in a local directory. * [Tree-sitter](/integrations/tree-sitter/): chunking, outlines and metadata for source code. The other integrations are covered on the page for the feature they serve. The Guide column in the matrix links to each one: DuckDB, Redis and Redb with storage and caching, Kafka, Fluvio, Parquet and scraping with loading, MCP with agents, tiktoken with model usage, and Langfuse and metrics with observability. ## Reference * [`swiftide::integrations`](https://docs.rs/swiftide/0.34/swiftide/integrations/index.html) * [`swiftide` crate root, Cargo Features section](https://docs.rs/swiftide/0.34/swiftide/index.html#cargo-features) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html), [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) and [`StructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.StructuredPrompt.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) and [`SparseEmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SparseEmbeddingModel.html) * [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html), [`Retrieve`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Retrieve.html) and [`NodeCache`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.NodeCache.html) * [`Loader`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Loader.html) and [`TransformResponse`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.TransformResponse.html) * [`swiftide::agents::tools::mcp`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/mcp/index.html) * [`swiftide::langfuse`](https://docs.rs/swiftide/0.34/swiftide/langfuse/index.html) * [`swiftide::indexing::transformers`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/index.html) * [Feature flags](/reference/feature-flags/) * [Crates](/reference/crates/) ## Next [OpenAI and Azure](/integrations/openai/) # PgVector > Store nodes in Postgres with the pgvector extension, search them by cosine distance, filter on a metadata column or run your own SQL. Your indexing pipeline writes nodes into a Postgres table, and your query pipeline finds them by cosine distance. The integration sits behind the `pgvector` feature flag. The excerpts on this page also embed with FastEmbed, prompt through Ollama or use OpenAI, so add the feature flags for the clients you use: ```sh cargo add swiftide@0.34 --features pgvector,fastembed,ollama,openai ``` ## Running Postgres Any Postgres with the pgvector extension works. Swiftide’s own tests use the `pgvector/pgvector:pg17` image: ```sh docker run -d --name swiftide-pgvector -p 5432:5432 \ -e POSTGRES_USER=myuser -e POSTGRES_PASSWORD=mypassword -e POSTGRES_DB=mydatabase \ pgvector/pgvector:pg17 ``` The connection URL for this container is `postgresql://myuser:mypassword@localhost:5432/mydatabase`. ## Builder examples/index_md_into_pgvector.rs ```rust let (_pgv_db_container, pgv_db_url) = swiftide_test_utils::start_postgres().await; tracing::info!("pgv_db_url :: {:#?}", pgv_db_url); let llm_client = integrations::ollama::Ollama::default() .with_default_prompt_model("llama3.2:latest") .to_owned(); let fastembed = integrations::fastembed::FastEmbed::try_default().expect("Could not create FastEmbed"); // Configure Pgvector with a default vector size, a single embedding // and in addition to embedding the text metadata, also store it in a field let pgv_storage = PgVector::builder() .db_url(pgv_db_url) .vector_size(384) .with_vector(EmbeddedField::Combined) .with_metadata(METADATA_QA_TEXT_NAME) .table_name("swiftide_pgvector_test".to_string()) .build() .unwrap(); ``` `start_postgres` is a test helper from `swiftide-test-utils`. That crate is not published, so you cannot add it as a dependency. It starts the same container through testcontainers and returns its URL. In your application, pass your own URL to `db_url`. The example uses the `fastembed` and `ollama` feature flags. `MetadataQAText` calls `llama3.2` through a local Ollama, so pull that model first. `db_url` and `vector_size` are required, and `build()` returns an error without them. `with_vector` is required in practice: without a vector field, setup fails with `No vector field found in configuration`. `vector_size` must match your embedder: FastEmbed’s default model returns 384 dimensions. `with_metadata` adds a metadata column. The example stores the questions and answers that `MetadataQAText` generates, under the field `Questions and Answers (text)`. | Setting | Default | | --------------------- | --------------------- | | `table_name` | `swiftide_pgv_store` | | `batch_size` | 50 nodes per insert | | `db_max_connections` | 10 | | `db_max_retry` | 3 connection attempts | | `db_conn_retry_delay` | 3 seconds | `build()` does not connect. The connection pool opens on first use, retrying as configured above. Cloning a `PgVector` is cheap, and the clones share that pool. In an application, read the URL from your configuration: ```rust let pgvector = PgVector::builder() .db_url(std::env::var("DATABASE_URL")?) .vector_size(1536) .with_vector(EmbeddedField::Combined) .with_metadata("path") .table_name("docs_chunks") .build()?; indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) // Every node needs each `with_metadata` field, or storing it fails .then(|mut node: TextNode| -> anyhow::Result { let path = node.path.to_string_lossy().into_owned(); node.metadata.insert("path", path); Ok(node) }) .then_in_batch(Embed::new(openai.clone())) .then_store_with(pgvector.clone()) .run() .await?; ``` `1536` matches OpenAI’s `text-embedding-3-small`, so build the client with `.default_embed_model("text-embedding-3-small")`. The closure transformer copies the file path into a `path` metadata field, which the [filter](#filters) below uses. ## Table layout When the indexing pipeline starts, the persist setup runs `CREATE EXTENSION IF NOT EXISTS vector`, creates the table if it is missing, and adds a Hierarchical Navigable Small World (HNSW) index. For the example above, that comes down to: ```sql CREATE TABLE IF NOT EXISTS swiftide_pgvector_test ( id UUID NOT NULL, chunk TEXT NOT NULL, vector_combined VECTOR(384), meta_questions_and_answers JSONB, PRIMARY KEY (id) ); CREATE INDEX IF NOT EXISTS swiftide_pgvector_test_embedding_idx ON swiftide_pgvector_test USING hnsw (vector_combined vector_cosine_ops); ``` * `id` is the node id, a universally unique identifier (UUID) derived from the node’s path and chunk. * `chunk` holds the chunk text. * Each `with_vector` field becomes a `vector_` column: `EmbeddedField::Combined` becomes `vector_combined`, `EmbeddedField::Chunk` becomes `vector_chunk`. * Each `with_metadata` field becomes a `meta_` JSONB column. It holds an object with the original field name as key, here `{"Questions and Answers (text)": "..."}`. Column names are normalized. Swiftide keeps the text before the first `(`, `[`, `{` or `<`, takes up to three words, joins them with underscores, lowercases them and drops everything except letters, digits and underscores. The table name must start with a letter or underscore, contain only letters, digits, `_` or `$`, and not be a reserved word such as `table`. It can be at most 49 bytes, because setup appends `_embedding_idx` to name the index and Postgres identifiers stop at 63 bytes. Otherwise setup fails with `Invalid table or field name`. The table is created only when it is missing. An existing table is kept as it is, so a new vector size or a new column means dropping the table first. The example drops it before every run for that reason. `get_pool()` returns the `sqlx` pool for your own Structured Query Language (SQL), such as dropping the table or deleting stale rows, and `get_table_name()` returns the table name. ## Storage examples/index_md_into_pgvector.rs ```rust tracing::info!("Starting indexing pipeline"); indexing::Pipeline::from_loader(FileLoader::new(test_dataset_path).with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(10..2048)) .then(MetadataQAText::new(llm_client.clone())) .then_in_batch(Embed::new(fastembed.clone()).with_batch_size(100)) .then_store_with(pgv_storage.clone()) .run() .await?; ``` `then_store_with` hands nodes to PgVector in batches of `batch_size`. Each batch is one `INSERT ... SELECT FROM UNNEST(...)` statement inside a transaction, with `ON CONFLICT (id) DO UPDATE`. Indexing the same chunk again overwrites its row. A changed chunk gets a new id, so its old row stays until you delete it. The default embed mode produces one `Combined` vector per node, which matches `with_vector(EmbeddedField::Combined)`. ## Retrieval examples/index_md_into_pgvector.rs ```rust async fn ask_query( llm_client: impl SimplePrompt + Clone + 'static, embed: FastEmbed, vector_store: PgVector, questions: Vec, ) -> Result, Box> { // By default the search strategy is SimilaritySingleEmbedding // which takes the latest query, embeds it, and does a similarity search // // Pgvector will return an error if multiple embeddings are set // // The pipeline generates subquestions to increase semantic coverage, embeds these in a single // embedding, retrieves the default top_k documents, summarizes them and uses that as context // for the final answer. let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::GenerateSubquestions::from_client( llm_client.clone(), )) .then_transform_query(query_transformers::Embed::from_client(embed)) .then_retrieve(vector_store.clone()) .then_transform_response(response_transformers::Summary::from_client( llm_client.clone(), )) .then_answer(answers::Simple::from_client(llm_client.clone())); let results: Vec = pipeline .query_all(questions) .await? .iter() .map(|result| result.answer().to_string()) .collect(); Ok(results) } ``` `query::Pipeline::default()` uses `SimilaritySingleEmbedding` with a `top_k` of 10. Embed the query with the same model you indexed with. PgVector selects `id`, `chunk` and every metadata column, orders by cosine distance (`<=>`) and returns `top_k` documents. Each document carries its metadata under the original field names. Retrieval needs exactly one vector column. With more than one, it fails with `Search strategy for multiple vector fields in the schema is not yet implemented`. PgVector has no hybrid search. ## Filters `SimilaritySingleEmbedding` takes a filter of the form `key = value`: ```rust // `FileLoader::new("docs")` stores paths with the `docs/` prefix, and the match is exact let search = SimilaritySingleEmbedding::from_filter("path = docs/README.md".to_string()).with_top_k(5); let pipeline = query::Pipeline::from_search_strategy(search) .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(pgvector) .then_answer(answers::Simple::from_client(openai)); ``` PgVector splits the filter at the first `=`, trims whitespace and strips double quotes around the value. It then adds `WHERE meta_ ->> $1 = $2`, binding the original key and the value. A filter is one exact text equality on a `with_metadata` field. The key must match the original field name, and the column must exist. The value must match the stored text exactly: `FileLoader::new("docs")` stores `docs/README.md`, not `README.md`. Nodes do not carry a `path` metadata field on their own; add one before storing, as in the [own database excerpt](#builder). ## Custom SQL For anything a filter cannot express, `CustomStrategy>` runs your own query: ```rust use sqlx::{Postgres, QueryBuilder}; // The default table is `swiftide_pgv_store`; `meta_path` exists only with `.with_metadata("path")` let table = pgvector.get_table_name().to_owned(); let keyword_search = CustomStrategy::from_query(move |query: &Query| { // The retriever reads `id`, `chunk` and every `meta_` column from each row let mut sql = QueryBuilder::::new(format!("SELECT id, chunk, meta_path FROM {table}")); sql.push(" WHERE to_tsvector('english', chunk) @@ plainto_tsquery('english', ") .push_bind(query.current().to_string()) .push(") LIMIT 10"); Ok(sql) }); let pipeline = query::Pipeline::from_search_strategy(keyword_search) .then_retrieve(pgvector) .then_answer(answers::Simple::from_client(openai)); ``` Query the PgVector table: take its name from `get_table_name()` rather than writing it by hand. PgVector maps each row by column name. `id` and `chunk` are required, and every column that starts with `meta_` becomes metadata. A custom strategy does not read the query embedding, so this pipeline needs no `Embed` step. See [Retrieval and search strategies](/querying/retrieval-and-search-strategies/) for how `CustomStrategy` builds the query and which `sqlx` version to add. ## Gotchas * Every node must carry every `with_metadata` field. A node without one fails the batch with `Missing metadata field`. * A node without the vector named in `with_vector` is stored as an empty vector, and Postgres rejects the batch with a dimension error. Keep the embed mode and `with_vector` in sync: `EmbedMode::PerField` produces no `Combined` vector. * Only the first vector column gets the HNSW index. * `SimilaritySingleEmbedding` fails with `Missing embedding in query state` when the query pipeline has no `Embed` query transformer. ## Reference * [`PgVector`](https://docs.rs/swiftide/0.34/swiftide/integrations/pgvector/struct.PgVector.html) * [`PgVectorBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/pgvector/struct.PgVectorBuilder.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html) * [`Retrieve`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Retrieve.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`CustomStrategy`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.CustomStrategy.html) * [`MetadataQAText`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQAText.html) * [`indexing::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) ## Next [LanceDB](/integrations/lancedb/) # Qdrant > Store indexed chunks in a Qdrant collection with dense and sparse vectors, then retrieve them with similarity or hybrid search and payload filters. With the `qdrant` feature flag, an indexing pipeline writes nodes into a Qdrant collection, and a query pipeline searches that collection by similarity or with hybrid search. ```sh cargo add swiftide@0.34 --features qdrant ``` Hybrid search also needs `fastembed` for sparse embeddings, and the filter snippet needs `openai`. ## Local instance and connection Start Qdrant in Docker: ```sh docker run -d -p 6333:6333 -p 6334:6334 qdrant/qdrant ``` Swiftide talks to Qdrant over gRPC Remote Procedure Call (gRPC) on port 6334, not over the Hypertext Transfer Protocol (HTTP) API on 6333. ## Builder examples/docs-qa/src/store.rs ```rust let qdrant = Qdrant::builder() .batch_size(BATCH_SIZE) .vector_size(DENSE_VECTOR_SIZE) // One dense and one sparse vector, both over the chunk *and* its metadata .with_vector(EmbeddedField::Combined) .with_sparse_vector(EmbeddedField::Combined) .collection_name(&collection.0) .build()?; ``` `Qdrant::builder()` reads the address from `QDRANT_URL`, falls back to `http://localhost:6334`, and sends `QDRANT_API_KEY` when set. `Qdrant::try_from_url(url)` returns `Result`, so add `?` before calling builder methods. The builder’s `client` method takes your own `qdrant_client::Qdrant`. `vector_size` is required and must match your embedder. docs-qa uses 384, the size of FastEmbed’s default dense model. The other settings have defaults: | Setting | Default | | ----------------- | ---------------------------------------------- | | `collection_name` | `swiftide` | | `vector_distance` | `Distance::Cosine` | | `batch_size` | 50 nodes per upsert | | vectors | one dense vector for `EmbeddedField::Combined` | `with_vector` adds a dense vector and `with_sparse_vector` adds a sparse one. The first `with_vector` call replaces the default, so list `EmbeddedField::Combined` yourself when you still need it. Both take an `EmbeddedField`, which names the text the vector was computed from. `Combined` is the chunk together with its metadata. ## Stored points `Qdrant` implements `Persist`, so you hand it to `then_store_with`: examples/docs-qa/src/index.rs ```rust indexing::Pipeline::from_loader(loader) .then_chunk(ChunkMarkdown::from_chunk_range( MIN_CHUNK_CHARS..MAX_CHUNK_CHARS, )) // A short title and keywords per chunk; both end up in the embedded text and help the // sparse (keyword) side of hybrid search in particular. .then( MetadataTitle::from_client(llm.clone()) .prompt_template(TITLE_PROMPT) .build()?, ) .then( MetadataKeywords::from_client(llm.clone()) .prompt_template(KEYWORDS_PROMPT) .build()?, ) .then_in_batch(Embed::new(embedders.dense.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_in_batch(SparseEmbed::new(embedders.sparse.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_store_with(qdrant.clone()) .run() .await ``` Before the first node, `setup` creates the collection if it is missing. An existing collection is kept as it is. The pipeline then upserts nodes in batches. Every point carries this payload: * each metadata field of the node, under its own name * `path`, the node’s source path * `content`, the chunk text * `last_updated_at`, the time of the upsert The point ID is a universally unique identifier (UUID) computed from the path and the chunk, so indexing the same chunk again overwrites its point. Debug builds wait for each upsert to apply; release builds do not. ### Captured output A debug build of docs-qa indexed four markdown pages into 43 chunks, with metadata from `claude-haiku-5-5` and embeddings from local FastEmbed models: ```text Indexed 43 chunks into `docs-qa` in 71.7s ``` ## Named vectors A point can hold one dense vector per embedded field: examples/store_multiple_vectors.rs ```rust indexing::Pipeline::from_loader(FileLoader::new("LICENSE")) .with_concurrency(1) .with_embed_mode(EmbedMode::PerField) .then_chunk(ChunkMarkdown::from_chunk_range(20..2048)) .then(MetadataQAText::new(openai_client.clone())) .then(MetadataSummary::new(openai_client.clone())) .then(MetadataTitle::new(openai_client.clone())) .then(MetadataKeywords::new(openai_client.clone())) .then_in_batch(Embed::new(openai_client.clone()).with_batch_size(10)) .log_all() .filter_errors() .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-multi-vectors") .with_vector(EmbeddedField::Chunk) .with_vector(EmbeddedField::Metadata(metadata_qa_text::NAME.into())) .with_vector(EmbeddedField::Metadata(metadata_summary::NAME.into())) .with_vector( VectorConfig::builder() .embedded_field(EmbeddedField::Metadata(metadata_title::NAME.into())) .distance(Distance::Manhattan) .build()?, ) .with_vector(EmbeddedField::Metadata(metadata_keywords::NAME.into())) .build()?, ) .run() .await?; ``` `EmbedMode::PerField` embeds the chunk and every metadata field separately. `EmbedMode::Both` adds the combined text. When the collection has named vectors, Qdrant stores only the ones you configured and skips the rest. The embed mode and the configured vectors must agree. A node with exactly one dense vector and no sparse vector is always sent as the unnamed vector. With several configured vectors, use `EmbedMode::PerField` or `Both`. With one configured vector, keep the default embed mode. Sparse vectors are only stored for fields that also have a configured dense vector. `VectorConfig::builder()` overrides the size or distance of one vector, here `Distance::Manhattan` for the title. With one dense vector and no sparse vector, the collection uses Qdrant’s unnamed vector. Otherwise vectors are named after their field: `Combined`, `Chunk` or `Metadata: `. Sparse vectors get a `_sparse` suffix, such as `Combined_sparse`. ## Similarity search A query pipeline searches Qdrant through `then_retrieve`. This one narrows the search with a payload filter: ```rust // Payload keys are the node's metadata plus `path`, `content` and `last_updated_at` let guides = Filter { must: vec![Condition::matches("path", ["README.md", "CONTRIBUTING.md"])], must_not: vec![Condition::matches("deprecated", true)], ..Default::default() }; let similarity = SimilaritySingleEmbedding::from_filter(guides.clone()).with_top_k(5); let pipeline = query::Pipeline::from_search_strategy(similarity) .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(openai)); // On a hybrid search, the filter applies to the dense and the sparse candidates let hybrid = HybridSearch::from_filter(guides).with_top_n(20); ``` `SimilaritySingleEmbedding` returns the 10 nearest chunks unless you set `with_top_k`. Only `then_retrieve(qdrant)` is Qdrant-specific. The query must already carry a dense embedding from `query_transformers::Embed`, made with the model you indexed with. Swiftide picks the vector to search from the configuration of the `Qdrant` value in the query pipeline, not from the collection. With more than one vector or any sparse vector configured, it searches the vector named `Combined`. Build the retriever with the same vector configuration as the indexing side; docs-qa calls the same `store::connect` for both. The collection in the previous section has no `Combined` vector, so it cannot serve this search. ## Hybrid search Hybrid search adds a sparse, keyword-like search to the dense one: examples/docs-qa/src/query.rs ```rust query::Pipeline::from_search_strategy( HybridSearch::default() .with_top_n(CANDIDATES) .with_top_k(CANDIDATES), ) ``` The query then needs both embeddings before retrieval: examples/docs-qa/src/query.rs ```rust pipeline // Hybrid search needs both a dense and a sparse embedding of the question .then_transform_query(query_transformers::Embed::from_client( self.embedders.dense.clone(), )) .then_transform_query(query_transformers::SparseEmbed::from_client( self.embedders.sparse.clone(), )) .then_retrieve(self.qdrant.clone()) // A local cross-encoder reorders the fused candidates; only the best few remain .then_transform_response(self.reranker.clone()) .then_answer(self.answerer.clone()) ``` Qdrant fetches `top_n` candidates from the dense and from the sparse vector and fuses them with Reciprocal Rank Fusion (RRF); see [search strategies](/querying/retrieval-and-search-strategies/) for `top_n` and `top_k`. Both vector fields default to `Combined`; `with_dense_vector_field` and `with_sparse_vector_field` select other named vectors. The collection needs a sparse vector for the same field. Without a sparse embedding on the query, retrieval fails with “No sparse embedding for query”. ## Payload filters Both search strategies take a Qdrant `Filter`, as in the similarity snippet. Swiftide re-exports the client crate at `swiftide::integrations::qdrant::qdrant_client`. Conditions match payload keys, including any metadata field. An array of strings matches any of its values as keywords. An owned `String` that contains whitespace becomes a full-text match instead of a keyword match. A full-text match needs a full-text payload index on that field, which Swiftide does not create. Without one, Qdrant matches the text as an exact substring. Qdrant implements hybrid search only for `HybridSearch`. Write that type even when you set no filter. For `SimilaritySingleEmbedding`, Qdrant accepts both `Filter` and no filter. `SimilaritySingleEmbedding::default()` is the search without a filter and needs no type annotation. Before 0.34, it needed one. ## Gotchas * Retrieved content is `Debug`-quoted, with escaped newlines. docs-qa wraps the retriever in [`CleanQdrant`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/retrieval.rs) (the `self.qdrant` in the query pipeline snippet above), which restores the text before reranking and answering. * A changed vector size, distance or set of vectors needs a new collection. * A changed chunk gets a new point ID, and the old point stays. docs-qa’s `--recreate` flag drops the collection first. * Retrieved documents keep every payload key except `content` as metadata. * `client()` returns a shared reference to the inner client, `&Arc`. docs-qa uses it for a health check, `collection_exists` and `delete_collection`. * If Qdrant is down, `run` fails during setup. See [unreachable stores](/reference/troubleshooting-and-faq/#unreachable-stores). ## Reference * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`QdrantBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.QdrantBuilder.html) * [`VectorConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.VectorConfig.html) * [`SparseVectorConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.SparseVectorConfig.html) * [`Distance`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/type.Distance.html) * [`qdrant_client`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/qdrant_client/index.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [`EmbedMode`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbedMode.html) * [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html) * [`Retrieve`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Retrieve.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`HybridSearch`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.HybridSearch.html) * [docs-qa source](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa) ## Next [PgVector](/integrations/pgvector/) # Tree-sitter > Split source code along syntax boundaries with ChunkCode and add outlines, questions, references and definitions as metadata, behind the tree-sitter feature flag. With this integration your indexing pipeline splits source files along functions, classes and other syntax boundaries instead of at a fixed length, and adds code-aware metadata to every chunk. Tree-sitter is a parser library with a grammar per language. Swiftide ships the grammars and a set of transformers that use them. ## Feature flag ```sh cargo add swiftide@0.34 --features tree-sitter ``` The `tree-sitter` feature flag adds the grammars and the transformers `ChunkCode`, `MetadataQACode`, `MetadataRefsDefsCode`, `OutlineCodeTreeSitter` and `CompressCodeOutline`. They are re-exported at `swiftide::indexing::transformers`, next to the other transformers. `SupportedLanguages` sits at `swiftide::integrations::treesitter`. The `tree-sitter-perl` feature flag adds Perl and turns on `tree-sitter` too. ## Supported languages ```rust // Case-insensitive: "python", "Python" and "PYTHON" all parse let language: SupportedLanguages = "python".parse()?; indexing::Pipeline::from_loader( FileLoader::new("src").with_extensions(language.file_extensions()), ) .then_chunk(ChunkCode::try_for_language(language)?) .then_store_with(MemoryStorage::default()) .run() .await?; ``` `SupportedLanguages` implements `FromStr` and ignores case. `ChunkCode`, `OutlineCodeTreeSitter` and `MetadataRefsDefsCode` take a `SupportedLanguages` value or a string such as `"rust"`, and return an error that says `Treesitter language not supported` for a name they do not know. `MetadataQACode` and `CompressCodeOutline` take no language, only an LLM client. `file_extensions()` gives the extensions to pass to `FileLoader`. | Language | Names | Extensions | Outline | References and definitions | | ---------- | ------------------------- | ---------------- | ------- | -------------------------- | | Rust | `rust` | rs | yes | yes | | TypeScript | `typescript` | ts, tsx, js, jsx | yes | yes | | JavaScript | `javascript` | js, jsx | yes | yes | | Python | `python` | py | yes | yes | | Ruby | `ruby` | rb | yes | yes | | Java | `java` | java | yes | yes | | C# | `csharp`, `c-sharp`, `c#` | cs, csx | yes | yes | | Go | `go` | go | no | yes | | Solidity | `solidity` | sol | no | yes | | C | `c` | c, h, o | no | no | | C++ | `cpp`, `c++` | c, h, o, cc, cpp | no | no | | Elixir | `elixir` | ex, exs | no | no | | HTML | `html` | html, htm, xhtml | no | no | | PHP | `php` | php | no | no | | Perl | `perl` | pm, pl, t | no | no | Every language works with `ChunkCode`. The last two columns say which languages `OutlineCodeTreeSitter` and `MetadataRefsDefsCode` handle. ## Chunk size and splitting ```rust indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["rs"])) .filter_cached(Redis::try_from_url(redis_url, "swiftide-examples")?) .then(MetadataQACode::new(openai_client.clone())) .then_chunk(ChunkCode::try_for_language_and_chunk_size( "rust", 10..2048, )?) .then_in_batch(Embed::new(openai_client.clone()).with_batch_size(10)) .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-examples") .build()?, ) .run() .await?; ``` The example also uses the `redis`, `qdrant` and `openai` feature flags, and needs a running Redis server for the node cache. See [feature flags](/reference/feature-flags/). `ChunkCode` is a chunker. It parses each file and walks the syntax tree, adding top-level items to a chunk. It closes the chunk when the next item would bring it to the maximum size or past it. An item larger than the maximum is split along its own children, one level down at a time. The chunk size is a `ChunkSize`. A number, such as `1000`, is a maximum in bytes. A range, such as `10..2048`, sets a minimum and a maximum: a chunk no larger than the minimum is dropped. `ChunkCode::try_for_language` uses a maximum of 1500 bytes. `with_concurrency` sets how many files are chunked at once. ## Several languages in one pipeline `ChunkCode` handles one language. To index several, pick a `ChunkCode` per file extension, as the ask-your-codebase showcase does with its `ChunkByLanguage` chunker (see [Chunking](/indexing/chunking/)). The first language that lists an extension wins, so the order of the list matters where extensions overlap. ## Questions per chunk examples/ask-your-codebase/src/index.rs ```rust let stats = indexing::Pipeline::from_loader( FileLoader::new(codebase.root()).with_extensions(&languages.extensions()), ) .filter(test_code::not_test_files(codebase.clone())) // Drop unit tests before the cache, which keys on path and content. Changing the content // after it would make the cache miss on every run. .then(test_code::without_unit_tests) .then(files_found.count()) // Skip files whose path and content are unchanged since the last successful run .filter_cached(codebase.node_cache()?) .then(files_changed.count()) .then(record_relative_path(codebase.clone())) // Split along syntax boundaries (functions, impls, ...) with tree-sitter .then_chunk(ChunkByLanguage::try_new(languages, CHUNK_SIZE)?) // Ask the LLM which questions each chunk answers; stored as metadata and embedded .then( MetadataQACode::from_client(llm) .prompt_template(QUESTIONS_PROMPT) .num_questions(QUESTIONS_PER_CHUNK) .concurrency(concurrent_requests) .build()?, ) // Embed chunk + metadata locally, in batches .then_in_batch(Embed::new(embedding::local_model()?).with_batch_size(embedding::BATCH_SIZE)) .then_store_with(codebase.vector_store()?) .run() .await?; ``` The showcase builds swiftide with the `tree-sitter`, `fastembed`, `lancedb`, `redb` and `openai` feature flags. `MetadataQACode` asks a large language model (LLM) which questions the code answers and stores the reply under the metadata key `Questions and Answers (code)`. A question at query time then finds the chunk more easily, because the questions are embedded with the code. Run it before chunking to send one prompt per file, which every chunk of that file then inherits. Run it after chunking to get questions per chunk, at one prompt per chunk. It writes 5 questions by default. `MetadataQACode::new(client)` builds it with the defaults, and `from_client(client)` returns a builder for `prompt_template`, `num_questions` and `concurrency`. The showcase runs it after chunking, so each chunk gets its own questions. It also replaces the default prompt with one that asks about behaviour, defined at the top of the same file. When a node has `Outline` metadata, the default prompt includes the outline. The showcase’s run, with `claude-haiku-5-5` writing the questions and FastEmbed embedding on the CPU, indexed the `swiftide-agents` crate like this: ```text Indexed 34 chunks from 14 new or changed files (14 files found, 0 unchanged) in 91.9s ``` ## Outlines, references and definitions `OutlineCodeTreeSitter`, `CompressCodeOutline` and `MetadataRefsDefsCode` give each chunk an outline of its file and the symbols it defines and uses. For pipelines that add them to every chunk, see [Code-aware indexing](/indexing/code-aware-indexing/). ## Gotchas * `OutlineCodeTreeSitter` and `MetadataRefsDefsCode` build for every language, but panic on a node in a language they do not handle. `OutlineCodeTreeSitter` skips nodes below its minimum file size first. The panic happens inside a spawned task, so `run()` fails with a join error instead of crashing the process. Check the table before you use them. * When a file has a syntax error, `ChunkCode` logs a warning and returns the whole file as one chunk, whatever its size. `OutlineCodeTreeSitter` returns an error for that node, and the first error stops the whole pipeline: `run()` fails. Add `filter_errors()` after it when your codebase may contain files that do not parse. See [Streams and concurrency](/indexing/streams-and-concurrency/). * `SupportedLanguages::Typescript` lists `tsx`, `js` and `jsx` in `file_extensions()`, but uses the plain TypeScript grammar, which cannot parse JSX. A `.tsx` or `.jsx` file that contains JSX has a syntax error, with the consequences above. Put JavaScript before TypeScript when you pick a language by extension, so `js` and `jsx` go to the JavaScript grammar. * The C and C++ lists both include `c`, `h` and `o`. `o` is the extension of compiled object files, so a `FileLoader` built from them picks up build output. * `MetadataQACode` and `CompressCodeOutline` send one prompt per node. On a large codebase that is many requests; lower the concurrency or wrap the client in `LanguageModelWithBackOff` to stay within rate limits. ## Reference * [`SupportedLanguages`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/enum.SupportedLanguages.html) * [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/chunk_code/struct.ChunkCode.html) * [`ChunkSize`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/enum.ChunkSize.html) * [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/metadata_qa_code/struct.MetadataQACode.html) * [`MetadataRefsDefsCode`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/metadata_refs_defs_code/struct.MetadataRefsDefsCode.html) * [`OutlineCodeTreeSitter`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/outline_code_tree_sitter/struct.OutlineCodeTreeSitter.html) * [`CompressCodeOutline`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/compress_code_outline/struct.CompressCodeOutline.html) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [Code-aware indexing](/indexing/code-aware-indexing/) ## Next [Ask your codebase](/showcases/ask-your-codebase/)