Retrieval and search strategies

A search strategy decides how many documents come back and how a retriever finds them. You pick a strategy, set top_k and a filter, and hand it to your storage, which acts as the retriever: Qdrant, PgVector, LanceDB or DuckDB.

Similarity search

let search = SimilaritySingleEmbedding::default().with_top_k(top_k);

let pipeline = query::Pipeline::from_search_strategy(search)
    // Rephrase the question into several, to widen what the search can match
    .then_transform_query(
        GenerateSubquestions::builder()
            .client(llm.clone())
            .prompt_template(SUBQUESTIONS_PROMPT.into())
            .build()?,
    )
    // Embed with the same local model that embedded the chunks
    .then_transform_query(Embed::from_client(embedding::local_model()?))
    .then_retrieve(codebase.vector_store()?)
    // Answer the original question from the retrieved code, each chunk under its file path
    .then_answer(
        answers::Simple::builder()
            .client(llm)
            .prompt_template(ANSWER_PROMPT.into())
            .document_template(CHUNK_TEMPLATE)
            .build()?,
    );

let answered = pipeline.query(question).await?;

Source: examples/ask-your-codebase/src/query.rs#L36-L58

SimilaritySingleEmbedding searches with the dense embedding that the Embed query transformer put on the query. It returns top_k documents. The default is 10. ask-your-codebase searches a LanceDB table and sets top_k from its --top-k flag, which defaults to 20.

with_top_k takes the strategy by value and returns it, so it chains onto default() or from_filter. SimilaritySingleEmbedding::default() exists only for the unfiltered strategy, so its filter type is inferred as (), even on stores that also retrieve with a typed filter. For a filtered search, start from from_filter or call with_filter, as in Filters.

Hybrid search

query::Pipeline::from_search_strategy(
    HybridSearch::default()
        .with_top_n(CANDIDATES)
        .with_top_k(CANDIDATES),
)

Source: examples/docs-qa/src/query.rs#L108-L112

HybridSearch combines a keyword or sparse search with a dense search. top_n caps the candidates of each search. top_k caps the documents that come back. Both default to 10. docs-qa fetches 20 candidates and later reranks them down to 10. with_top_n, with_top_k, with_dense_vector_field and with_sparse_vector_field take the strategy by value and return it, so they chain without .to_owned().

HybridSearch implements Default for every filter type. When nothing else fixes the filter type, name it, as docs-qa does with HybridSearch<Filter> in its return type, or the compiler stops with E0283.

On Qdrant, the query needs a dense and a sparse embedding, so the pipeline runs both query transformers before it retrieves. The example creates two FastEmbed models in its setup: fastembed for dense and fastembed_sparse for sparse embeddings.

// Use sophisticated model for our query
let openai = openai::OpenAI::builder()
    .default_prompt_model("gpt-4o")
    .build()
    .unwrap();

let query_pipeline = query::Pipeline::from_search_strategy(
    // Return a large amount of documents because we have a large context window
    // By default it uses the Combined fields, no need to configure
    HybridSearch::default().with_top_n(20).with_top_k(20),
)
// Generate subquestions on the initial query to increase our query coverage
.then_transform_query(query_transformers::GenerateSubquestions::from_client(
    openai.clone(),
))
// Generate the same embeddings we used for indexing
.then_transform_query(query_transformers::Embed::from_client(fastembed.clone()))
.then_transform_query(query_transformers::SparseEmbed::from_client(
    fastembed_sparse.clone(),
))
.then_retrieve(qdrant.clone())
// Answer with Simple, which either takes the documents as is (in this case), or any
// transformations applied after querying
.then_answer(answers::Simple::from_client(openai.clone()));

Source: examples/hybrid_search.rs#L65-L88

FastEmbed is the only sparse embedding model in Swiftide, so Qdrant hybrid search needs the qdrant and fastembed feature flags. The chunks need sparse embeddings too. Run the SparseEmbed transformer in the indexing pipeline next to Embed:

indexing::Pipeline::from_loader(FileLoader::new("swiftide-core/").with_extensions(&["rs"]))
    // Chunk fairly large as the context window is big
    .then_chunk(ChunkCode::try_for_language_and_chunk_size(
        "rust",
        10..2048,
    )?)
    // Generate metadata on the code chunks to increase our chances of finding the right code
    .then(MetadataQACode::from_client(openai.clone()).build().unwrap())
    .then_in_batch(
        transformers::SparseEmbed::new(fastembed_sparse.clone()).with_batch_size(batch_size),
    )
    .then_in_batch(transformers::Embed::new(fastembed.clone()).with_batch_size(batch_size))
    .then_store_with(qdrant.clone())
    .run()
    .await?;

Source: examples/hybrid_search.rs#L47-L61

Qdrant runs a sparse and a dense search, each limited to top_n. It merges them with reciprocal rank fusion (RRF) and keeps top_k. The collection needs a sparse vector next to the dense one; see Qdrant.

DuckDB runs hybrid search without a sparse embedding. It scores the chunk text with BM25 full-text search over query.current() and the dense embedding with cosine similarity. Each score is normalized to its maximum, and the final score is 0.8 times the dense score plus 0.2 times the full-text score. After GenerateSubquestions, the keyword text is the whole list of generated questions.

Filters

A filter narrows the search to documents with matching metadata. Each storage takes its own filter type. Qdrant takes a Filter from its client, which Swiftide re-exports at swiftide::integrations::qdrant::qdrant_client:

let only_readme = Filter::must([Condition::matches("path", "README.md".to_string())]);

let pipeline = query::Pipeline::from_search_strategy(SimilaritySingleEmbedding::from_filter(
    only_readme.clone(),
))
.then_transform_query(query_transformers::Embed::from_client(openai.clone()))
.then_retrieve(qdrant)
.then_answer(answers::Simple::from_client(openai));

// The same filter on a hybrid search applies to the dense and the sparse search
let _hybrid = HybridSearch::from_filter(only_readme).with_top_n(20);

Source: snippets/src/querying/retrieval.rs#L24-L34

PgVector and LanceDB take a String. PgVector expects key = value and compares it with the metadata column meta_<key>, so the key must be a metadata field on the table. Swiftide does not put the node path into metadata. To filter on path, insert it into each node’s metadata while you index (see Transformers and metadata), as ask-your-codebase does. Without it, PgVector refuses to store the node with “Missing metadata field”.

// `path` must be a metadata field on the table: `PgVector::builder().with_metadata("path")`.
// Swiftide does not add it for you; insert `path` into each node's metadata while indexing.
let search =
    SimilaritySingleEmbedding::from_filter("path = README.md".to_string()).with_top_k(5);

let pipeline = query::Pipeline::from_search_strategy(search)
    .then_transform_query(query_transformers::Embed::from_client(openai.clone()))
    .then_retrieve(pgvector)
    .then_answer(answers::Simple::from_client(openai));

Source: snippets/src/querying/retrieval.rs#L44-L52

LanceDB passes the string to its only_if method, which takes a SQL predicate on the table’s columns, such as path = 'README.md'. That column has a value only if you put path in the node metadata and configured it with with_metadata("path"). DuckDB has no filter.

Custom strategies

CustomStrategy hands the query to your closure, and the closure builds the query the storage runs. from_query takes a synchronous closure, from_async_query an async one. PgVector runs a sqlx::QueryBuilder<Postgres>, LanceDB a LanceDB query, and DuckDB a SQL string.

use sqlx::{Postgres, QueryBuilder};

// The default table is `swiftide_pgv_store`; `meta_path` exists only with `.with_metadata("path")`
let table = pgvector.get_table_name().to_owned();

let keyword_search = CustomStrategy::from_query(move |query: &Query<states::Pending>| {
    // The retriever reads `id`, `chunk` and every `meta_` column from each row
    let mut sql =
        QueryBuilder::<Postgres>::new(format!("SELECT id, chunk, meta_path FROM {table}"));
    sql.push(" WHERE to_tsvector('english', chunk) @@ plainto_tsquery('english', ")
        .push_bind(query.current().to_string())
        .push(") LIMIT 10");
    Ok(sql)
});

let pipeline = query::Pipeline::from_search_strategy(keyword_search)
    .then_retrieve(pgvector)
    .then_answer(answers::Simple::from_client(openai));

Source: snippets/src/querying/retrieval.rs#L62-L79

PgVector maps each row by column name. Select id, chunk and the meta_ columns you need. get_table_name() returns the table, swiftide_pgv_store unless you set another. Swiftide does not re-export sqlx, so add it yourself:

cargo add sqlx@0.9 --no-default-features --features postgres

A LanceDB custom strategy returns a query that implements lancedb::query::ExecutableQuery. Swiftide does not re-export lancedb either. Enable the lancedb feature flag and add the crate at the version Swiftide uses:

cargo add lancedb@0.33

DuckDB reads the first three columns of each row by position: the id, the chunk and the path.

Retrievers per storage

Storage (feature flag)	Strategies	Filter	Limits
`Qdrant` (`qdrant`)	`SimilaritySingleEmbedding`, `HybridSearch<Filter>`	Qdrant `Filter`	Similarity search uses the combined vector when the collection has more than one vector or a sparse vector
`PgVector` (`pgvector`)	`SimilaritySingleEmbedding`, `CustomStrategy<QueryBuilder<Postgres>>`	`String`, `key = value`	One vector field; orders by cosine distance
`LanceDB` (`lancedb`)	`SimilaritySingleEmbedding`, `CustomStrategy<Q: ExecutableQuery>`	`String`, SQL predicate	Exactly one vector field; content comes from the `chunk` column
`Duckdb` (`duckdb`)	`SimilaritySingleEmbedding`, `HybridSearch`, `CustomStrategy<String>`	none	Uses one vector, picked arbitrarily when several are configured; documents carry `id` and `path` metadata only

Setup for each storage is on Qdrant, PgVector and LanceDB. DuckDB has no integration page; see the Duckdb reference below.

Custom retrievers

A retriever implements Retrieve<S> for a search strategy S. It receives the strategy and the pending query, and returns query.retrieved_documents(documents). retrieved_documents adds the documents and records a retrieval in the query’s transformation history. The trait requires Clone.

/// Keeps the documents that share a word with the question. No store, no embeddings.
#[derive(Clone)]
struct KeywordRetriever {
    documents: Vec<Document>,
}

#[async_trait]
impl Retrieve<SimilaritySingleEmbedding> for KeywordRetriever {
    async fn retrieve(
        &self,
        search_strategy: &SimilaritySingleEmbedding,
        query: Query<states::Pending>,
    ) -> anyhow::Result<Query<states::Retrieved>> {
        let words: Vec<String> = query
            .current()
            .split_whitespace()
            .map(str::to_lowercase)
            .collect();

        let matches = self
            .documents
            .iter()
            .filter(|document| {
                let content = document.content().to_lowercase();
                words.iter().any(|word| content.contains(word.as_str()))
            })
            .take(usize::try_from(search_strategy.top_k())?)
            .cloned()
            .collect();

        Ok(query.retrieved_documents(matches))
    }
}

Source: snippets/src/querying/retrieval.rs#L87-L119

let retriever = KeywordRetriever {
    documents: vec![
        Document::from("Swiftide indexes data with an indexing pipeline."),
        Document::from("A query pipeline answers questions over indexed data."),
    ],
};

let pipeline = query::Pipeline::default()
    .then_retrieve(retriever)
    .then_answer(answers::Simple::from_client(openai));

Source: snippets/src/querying/retrieval.rs#L125-L134

A closure works too: then_retrieve accepts any Fn(&S, Query<states::Pending>) -> Result<Query<states::Retrieved>> that is Send + Sync + Clone. The closure is synchronous.

Gotchas

Retrieval fails when the query has no embedding. Qdrant and LanceDB report “No embedding for query”, PgVector and DuckDB “Missing embedding in query state”. Qdrant hybrid search also needs a sparse embedding (“No sparse embedding for query”).
HybridSearchBuilder::default() sets top_k and top_n to 0. Start from HybridSearch::default(), which sets both to 10.
Qdrant only implements hybrid search for HybridSearch<Filter>. Write that type in signatures, as docs-qa does, even without a filter.
Qdrant returns each document’s content Debug-quoted: wrapped in " with escaped newlines. docs-qa wraps the retriever in CleanQdrant (Qdrant content workaround), which restores the text before reranking and answering.
DuckDB hybrid search does not rank before it cuts. Each search takes the first top_n rows of the table, unranked, and only rows in both lists get scored. On a table larger than top_n, most chunks are never considered. Set top_n to at least the row count, or use SimilaritySingleEmbedding.
A small top_k hides relevant chunks. In ask-your-codebase, one answer had to infer a detail because the chunk that held it was not among the 20 retrieved. Raise top_k for broad questions, or retrieve more and rerank.

Reference

SimilaritySingleEmbedding
HybridSearch, HybridSearchBuilder
CustomStrategy
Retrieve
SearchStrategy
SparseEmbed, FastEmbed
query::Pipeline
Qdrant, PgVector, LanceDB, Duckdb
Showcases: ask-your-codebase, docs-qa

Reranking and response transformers