Transforming queries

A query pipeline can rewrite the user’s question before the retriever runs: a large language model (LLM) widens it into related questions, and an embedding model turns the result into the vectors your chunks were stored with.

The query transformers need no feature flag. The snippets use the OpenAI client and Qdrant, which sit behind the openai and qdrant feature flags:

cargo add swiftide@0.34 --features openai,qdrant
cargo add async-trait anyhow

The second line is for the custom query transformer further down, which uses the #[async_trait] attribute and anyhow::Result.

Subquestions

let pipeline = query::Pipeline::default()
    .then_transform_query(
        GenerateSubquestions::builder()
            .client(openai.clone())
            .num_questions(3)
            .build()?,
    )
    // Embed the text that the step above produced, with the model the chunks were embedded with
    .then_transform_query(Embed::from_client(embedder))
    .then_retrieve(qdrant)
    .then_answer(answers::Simple::from_client(openai));

Source: snippets/src/querying/transforming_queries.rs#L21-L31

A query transformer runs before retrieval. It takes the pending query and returns a new pending query. You add one with then_transform_query, as often as you need, and they run in the order you add them.

GenerateSubquestions sends the current query to an LLM through SimplePrompt. It asks for num_questions related questions, 5 by default. The reply replaces the current query, so the next step sees one text: the original question and its subquestions as a list. GenerateSubquestions::from_client builds it with the default prompt and count.

The default prompt is written for code. It asks the model to think about the dependencies, usage and structure of the code. Replace it when you search other content.

Subquestion prompts

The ask-your-codebase showcase sets its own prompt on the builder:

let search = SimilaritySingleEmbedding::default().with_top_k(top_k);

let pipeline = query::Pipeline::from_search_strategy(search)
    // Rephrase the question into several, to widen what the search can match
    .then_transform_query(
        GenerateSubquestions::builder()
            .client(llm.clone())
            .prompt_template(SUBQUESTIONS_PROMPT.into())
            .build()?,
    )
    // Embed with the same local model that embedded the chunks
    .then_transform_query(Embed::from_client(embedding::local_model()?))
    .then_retrieve(codebase.vector_store()?)
    // Answer the original question from the retrieved code, each chunk under its file path
    .then_answer(
        answers::Simple::builder()
            .client(llm)
            .prompt_template(ANSWER_PROMPT.into())
            .document_template(CHUNK_TEMPLATE)
            .build()?,
    );

let answered = pipeline.query(question).await?;

Source: examples/ask-your-codebase/src/query.rs#L36-L58

SUBQUESTIONS_PROMPT is a constant next to the pipeline:

/// Widens the question into the follow-up questions its answer involves.
///
/// They are phrased like the questions generated for each chunk while indexing, so the two
/// embed close together.
const SUBQUESTIONS_PROMPT: &str = r"
Your job is to help a search find the code that answers a developer's question about a codebase.

Question: {{ question }}

Write {{ num_questions }} short follow-up questions that the code answering this question would also
answer. Think about the mechanisms involved: the main loop, the types and enum variants, the
conditions that are checked, the functions that are called, and the configuration that changes
the outcome.

## Constraints
* Phrase them the way a developer would ask about this codebase, e.g. 'When does the agent stop?'.
* Do not guess names of types or functions.
* Respond with the original question and the follow-up questions only, one per line, prefixed
  with '- '.
";

27 collapsed lines
/// Renders a retrieved chunk under its file path, so the answer can cite it.
///
/// `metadata.path` is the `PATH_FIELD` metadata; a test keeps the two in step.
const CHUNK_TEMPLATE: &str = r"### {{ metadata.path }}

```
{{ content }}
```
";

/// Answers from the retrieved code and cites files.
const ANSWER_PROMPT: &str = r"
Answer a developer's question about a codebase, using snippets of its source code.

Question: {{ question }}

## Constraints
* Only use the snippets below. If they do not answer the question, say briefly what is missing.
* Be concrete: name the types, functions and variants involved and explain when they apply.
* Describe behaviour: what calls what, in which order, and under which conditions.
* Cite the file each statement comes from in backticks, e.g. (`src/lib.rs`).
* Use concise markdown. Add a short code example only if it shows how to use the API.

## Snippets

{{ documents }}
";

Source: examples/ask-your-codebase/src/query.rs#L65-L112

The field takes a Prompt, so convert a &'static str or String with .into() or Prompt::from. A borrowed &str built at runtime needs .to_string() first. The template receives two values: question, the current query, and num_questions.

The prompt phrases the subquestions like the questions that its indexing pipeline wrote for each chunk. Similar phrasing puts the query vector close to the chunk vectors. Keep the instruction to repeat the original question: the next step embeds only what the model returns. Prompts and templates covers the template syntax.

Dense and sparse embeddings

pipeline
    // Hybrid search needs both a dense and a sparse embedding of the question
    .then_transform_query(query_transformers::Embed::from_client(
        self.embedders.dense.clone(),
    ))
    .then_transform_query(query_transformers::SparseEmbed::from_client(
        self.embedders.sparse.clone(),
    ))
    .then_retrieve(self.qdrant.clone())
    // A local cross-encoder reorders the fused candidates; only the best few remain
    .then_transform_response(self.reranker.clone())
    .then_answer(self.answerer.clone())

Source: examples/docs-qa/src/query.rs#L121-L134

query_transformers::Embed embeds the current query with an EmbeddingModel and stores the vector in query.embedding. query_transformers::SparseEmbed does the same with a SparseEmbeddingModel and fills query.sparse_embedding. At this release only FastEmbed, behind the fastembed feature flag, implements SparseEmbeddingModel.

Use the models that your indexing pipeline embedded the chunks with. Vectors from different models do not compare. The docs-qa showcase creates its dense and sparse FastEmbed models together, and loads the same two models for its indexing and its query pipeline.

Similarity search needs embedding. Hybrid search on Qdrant needs both, and fails with No sparse embedding for query when the sparse one is missing. Retrieval and search strategies covers both strategies.

An embedding step reads the current query at the moment it runs. Put it after every step that rewrites the text. If Embed runs before GenerateSubquestions, the vector search never sees the subquestions.

Custom query transformers

/// Puts the instruction that BGE embedding models expect for search queries in front of the
/// question.
#[derive(Clone)]
pub struct BgeQueryInstruction;

#[async_trait]
impl TransformQuery for BgeQueryInstruction {
    async fn transform_query(
        &self,
        mut query: Query<states::Pending>,
    ) -> anyhow::Result<Query<states::Pending>> {
        let instructed = format!(
            "Represent this sentence for searching relevant passages: {}",
            query.current()
        );
        query.transformed_query(instructed);

        Ok(query)
    }
}

Source: snippets/src/querying/transforming_queries.rs#L53-L72

TransformQuery has one async method, transform_query. It takes the Query<states::Pending> by value and returns it. Implement it with the #[async_trait] attribute from the async-trait crate. The trait requires Send, Sync and DynClone, so derive Clone.

The current query has no setter of its own. transformed_query replaces it and records a Transformed event with the text before and after. The original question never changes. The fields embedding, sparse_embedding and documents are public, so a transformer that computes a vector sets it directly, as Embed does.

This example prepares a question for an embedding model of the BGE (BAAI General Embedding) family, which suggests an instruction in front of short search queries. The chunks are embedded without it.

let pipeline = query::Pipeline::default()
    // A closure is enough for a step without state
    .then_transform_query(|mut query: Query<states::Pending>| {
        let trimmed = query.current().trim().to_string();
        query.transformed_query(trimmed);
        Ok(query)
    })
    .then_transform_query(BgeQueryInstruction)
    .then_transform_query(Embed::from_client(embedder))
    .then_retrieve(qdrant)
    .then_answer(answers::Simple::from_client(openai));

Source: snippets/src/querying/transforming_queries.rs#L82-L92

A closure is a query transformer too. It must be Fn(Query<states::Pending>) -> Result<Query<states::Pending>> and Clone, Send and Sync. Closures are synchronous, so use a struct for a step that awaits.

The pipeline logs Transformed query at debug level after each step, with the transformer’s name() and the new current query. name() defaults to the last path segment of the type name, so a closure logs a generic closure name instead of a useful one. Override name() on a struct to get a clearer log line.

Transformed query after retrieval

let answered = pipeline.query("How does an agent decide to stop?").await?;

for event in answered.history() {
    if let Some(documents) = event.documents() {
        println!(
            "retrieved {} documents for:\n{}",
            documents.len(),
            event.before()
        );
    } else {
        println!("rewrote:\n{}\ninto:\n{}", event.before(), event.after());
    }
}

Source: snippets/src/querying/transforming_queries.rs#L35-L47

Since 0.33 the current query keeps the transformed text after retrieval. The Retrieved event records the current query at the moment of retrieval, as both before() and after(), and the documents it returned. Before 0.33, retrieval cleared the current query.

That text is not always what the search used. A vector retriever such as Qdrant searches with query.embedding, which Embed computed when it ran. A step after Embed that rewrites the text changes the event, not the vector.

history() lists the events in order. Each transformed_query call adds a Transformed event, and each retrieval adds a Retrieved event. Embed and SparseEmbed add nothing, because they leave the text alone.

Once the query is answered, current() returns the answer. Read the transformed query from the history instead.

Generating answers shows how the answer generator uses the original and the transformed query.

Gotchas

Two types are called Embed. query_transformers::Embed belongs in a query pipeline, indexing::transformers::Embed in an indexing pipeline. The same goes for SparseEmbed. Import them by module path.
After GenerateSubquestions, Embed turns the whole list into one vector. The pipeline does not run one search per subquestion.
DuckDB hybrid search, behind the duckdb feature flag, uses the subquestion list as its keyword text; see Retrieval and search strategies.
The FastEmbed Rerank response transformer scores documents against original(), and the Ragas evaluator matches rows by original(). Rewriting the question changes neither.
Query::new sets only the original question and leaves the current query empty, so an embedding step would embed an empty string. Pass a &str or String to query, or build the query with Query::from, which sets both.

Reference

Pipeline (then_transform_query)
TransformQuery
GenerateSubquestions
query_transformers::Embed
query_transformers::SparseEmbed
Query
TransformationEvent
states::Pending
EmbeddingModel
SparseEmbeddingModel
SimplePrompt
Prompt
answers::Simple
Showcases: ask-your-codebase and docs-qa

Retrieval and search strategies