# Reranking and response transformers

> Reorder and trim retrieved documents with the FastEmbed Rerank cross-encoder, condense them into an LLM Summary that the answer generator uses instead, or filter them with your own TransformResponse.

Your query pipeline fetches twenty candidate documents, keeps the ten a local reranker scores highest, and answers from those. You can also summarize the documents with a large language model (LLM) or filter them with your own response transformer.

## FastEmbed reranker

The docs-qa showcase builds its reranker next to its answer generator:

```rust title="examples/docs-qa/src/query.rs"
let reranker = Rerank::builder()
    .top_k(CONTEXT_CHUNKS)
    .document_template(Some(RERANK_TEMPLATE.to_string()))
    .build()?;

let answerer = answers::Simple::builder()
    .client(llm)
    .prompt_template(ANSWER_PROMPT.into())
    .document_template(DOCUMENT_TEMPLATE)
    .build()?;
```

Source: [`examples/docs-qa/src/query.rs#L73-L82`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L73-L82)

It then places the reranker between retrieval and the answer:

```rust title="examples/docs-qa/src/query.rs"
pipeline
    // Hybrid search needs both a dense and a sparse embedding of the question
    .then_transform_query(query_transformers::Embed::from_client(
        self.embedders.dense.clone(),
    ))
    .then_transform_query(query_transformers::SparseEmbed::from_client(
        self.embedders.sparse.clone(),
    ))
    .then_retrieve(self.qdrant.clone())
    // A local cross-encoder reorders the fused candidates; only the best few remain
    .then_transform_response(self.reranker.clone())
    .then_answer(self.answerer.clone())
```

Source: [`examples/docs-qa/src/query.rs#L121-L134`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L121-L134)

`Rerank` is a response transformer. A response transformer runs after `then_retrieve` and before `then_answer`. It takes the query in the `Retrieved` state and returns it in the same state.

`Rerank` runs a cross-encoder: a model that reads the question and one document together and returns a relevance score. That is slower than comparing embeddings, so you run it only on the candidates the retriever already found.

The builder has four fields:

- `top_k`: how many documents to keep, as a `usize`. The default is 10.
- `document_template`: an optional Tera template that renders each document before scoring. The setter takes an `Option<String>`, so pass `Some(..)`.
- `model`: the FastEmbed `TextRerank` model, wrapped in `Arc<tokio::sync::Mutex<_>>`. The default is BGE reranker base.
- `model_batch_size`: an optional batch size. The default is FastEmbed's own.

`Rerank` scores every document against `query.original()`, the question as the user asked it. Query transformers such as `GenerateSubquestions` replace `current()`, which the retriever searches with and which `answers::Simple` also sees. The reranker ignores `current()`. It replaces the query's documents with the `top_k` highest-scoring ones, best first.

`Rerank` lives in `swiftide::integrations::fastembed` behind the `fastembed` feature flag:

```sh
cargo add swiftide@0.34 --features fastembed
```

This is a docs-qa answer, written by `claude-sonnet-5-5` and reranked locally with BGE reranker base. The sources list the pages and sections of the kept documents, best first:

```text
Swiftide supports OpenAI and Azure OpenAI, Anthropic, Gemini, OpenRouter, AWS Bedrock Converse API, Groq, Ollama, and Dashscope. Integrations are feature-gated, so you enable a provider by adding its Cargo feature to `swiftide`.
...
Sources:
  [1] Swiftide › Swiftide capabilities for agents, typed task graphs, pipelines, and tracing  (README.md)
  [2] Swiftide › Swiftide: Rust framework for LLM agents, task graphs, and RAG  (README.md)
  [3] Swiftide › Swiftide Integrations Overview by Feature-Gated Category  (README.md)
  ...
  [9] Swiftide › Swiftide agent tool defined with the swiftide::tool macro  (README.md)
  [10] Swiftide › Installing Swiftide and Setting Up OpenAI Integration  (README.md)
```

The `openai` feature flag in that answer comes from the tenth source, the Quick Start chunk. With a `top_k` below 10 it would not have reached the model. Check a few answers before you lower `top_k`.

### Rerank templates

```rust title="examples/docs-qa/src/query.rs"
// The answer prompt receives the question and the rendered documents; each document is rendered
// with its page title and the chunk title written during indexing.
const ANSWER_PROMPT: &str = "You answer questions about Swiftide, a Rust library for building \
LLM applications, using excerpts from its documentation.

Question: {{ question }}

## Instructions
* Use only the documentation excerpts below. If they do not answer the question, say so plainly.
* Start with the direct answer in one or two sentences, then add supporting details.
* Only when the question asks how to do something, include the feature flags, commands or a \
short Rust snippet from the excerpts. Never invent APIs.
* Format the answer as markdown and keep it under 250 words. Do not refer to \"the excerpts\".

## Documentation excerpts

{{ documents }}
";

const DOCUMENT_TEMPLATE: &str = "### {{ metadata.Page | default(value=\"\") }}: \
{{ metadata.Title | default(value=\"\") }}
{{ content }}";

// The reranker scores the same title-first rendering. A chunk that is mostly code says little
// about what it shows, so its page and the title written during indexing go first.
const RERANK_TEMPLATE: &str = "{{ metadata.Page | default(value=\"\") }}: \
{{ metadata.Title | default(value=\"\") }}
{{ content }}";
```

Source: [`examples/docs-qa/src/query.rs#L30-L57`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L30-L57)

`RERANK_TEMPLATE` puts the page name and the chunk title, both written during indexing, before the content. A chunk that is mostly code says little about its topic, so the extra context helps it score fairly. The template only changes what the model scores. The documents stay as they were, and the answer generator renders them with its own `document_template`.

In docs-qa this template fixed one evaluation result. For "What is a task in Swiftide?", the chunk that lists what a task supports ranked 14th when the reranker scored only chunk text, so it was dropped. The answer missed two features, with answer recall 0.71 and context recall 0.57. With page and title in the template, the chunk made the kept ten and both scores reached 1.00. [Evaluating with Ragas](/querying/evaluation/#scores) explains these scores.

Model download, cache location and load failures are covered in [FastEmbed](/integrations/fastembed/).

### Minimal pipeline

The single-file example builds its reranker in its setup:

```rust
let openai_client = integrations::openai::OpenAI::builder()
    .default_prompt_model("gpt-4o")
    .build()?;

let fastembed = fastembed::FastEmbed::builder().batch_size(10).build()?;
let reranker = fastembed::Rerank::builder().top_k(5).build()?;

let qdrant = Qdrant::builder()
    .batch_size(50)
    .vector_size(384)
    .collection_name("swiftide-reranking")
    .build()?;
```

Source: [`examples/reranking.rs#L25-L36`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/reranking.rs#L25-L36)

It then generates subquestions for retrieval, while the reranker still scores against the original question:

```rust
// By default the search strategy is SimilaritySingleEmbedding
// which takes the latest query, embeds it, and does a similarity search
let pipeline = query::Pipeline::default()
    .then_transform_query(query_transformers::GenerateSubquestions::from_client(
        openai_client.clone(),
    ))
    .then_transform_query(query_transformers::Embed::from_client(fastembed.clone()))
    .then_retrieve(qdrant.clone())
    .then_transform_response(reranker)
    .then_answer(answers::Simple::from_client(openai_client.clone()));
```

Source: [`examples/reranking.rs#L49-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/reranking.rs#L49-L58)

## Summaries

```rust
// By default the search strategy is SimilaritySingleEmbedding
// which takes the latest query, embeds it, and does a similarity search
let pipeline = query::Pipeline::default()
    .then_transform_query(query_transformers::GenerateSubquestions::from_client(
        openai_client.clone(),
    ))
    .then_transform_query(query_transformers::Embed::from_client(
        openai_client.clone(),
    ))
    .then_retrieve(qdrant.clone())
    .then_transform_response(response_transformers::Summary::from_client(
        openai_client.clone(),
    ))
    .then_answer(answers::Simple::from_client(openai_client.clone()));
```

Source: [`examples/query_pipeline.rs#L39-L52`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/query_pipeline.rs#L39-L52)

`response_transformers::Summary` sends the retrieved documents to an LLM and asks for one summary of them. It stores the reply with `transformed_response`, which records a transformation event and replaces `query.current()`. It leaves `documents()` untouched.

`answers::Simple` then answers from the summary instead of the rendered documents; see [Generating answers](/querying/answering/). The documents stay on the query, so you can still list sources.

To change the instruction, pass your own `prompt_template`:

```rust
let summary = response_transformers::Summary::builder()
    .client(openai)
    .prompt_template(Prompt::from(
        "Summarize these excerpts as one markdown list. Keep every type name, \
         feature flag and command exactly as written.\n\
         {% for document in documents %}\n\
         ---\n\
         {{ document.metadata.path | default(value=\"unknown\") }}:\n\
         {{ document.content }}\n\
         {% endfor %}",
    ))
    .build()?;
```

Source: [`snippets/src/querying/reranking.rs#L75-L86`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/reranking.rs#L75-L86)

The template sees only `documents`, a list of documents with `content` and `metadata`. The question is not in its context.

You can chain response transformers. Put `Rerank` before `Summary` and the LLM summarizes only the kept documents.

## Custom response transformers

```rust
/// Keeps the first document of each file, in retrieval order, for at most `max` files.
#[derive(Clone)]
pub struct OnePerFile {
    pub max: usize,
}

#[async_trait]
impl TransformResponse for OnePerFile {
    async fn transform_response(
        &self,
        mut query: Query<states::Retrieved>,
    ) -> Result<Query<states::Retrieved>> {
        let mut seen_paths = HashSet::new();
        let documents = query.documents_mut();

        documents.retain(|document| {
            let path = document
                .metadata()
                .get("path")
                .and_then(|value| value.as_str())
                .unwrap_or_default();
            seen_paths.insert(path.to_string())
        });
        documents.truncate(self.max);

        Ok(query)
    }
}
```

Source: [`snippets/src/querying/reranking.rs#L15-L42`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/reranking.rs#L15-L42)

`TransformResponse` has one async method, `transform_response`. Implement it with `#[async_trait]`, from the `async-trait` crate or `swiftide::reexports::async_trait`. The trait requires `Clone` through `DynClone`, so derive it.

This transformer edits the list in place through `documents_mut`. It keeps the first document per `path` metadata value, in the order the documents arrive, and caps the list at `max`. Documents without a `path` share one empty key, so only the first of them is kept. It does not call `transformed_response`, so `answers::Simple` still renders the documents one by one. Call `transformed_response` only when your transformer produces text for the answer to use in place of the documents, as `Summary` does.

Place it after `then_retrieve`:

```rust
let pipeline = query::Pipeline::default()
    .then_transform_query(query_transformers::Embed::from_client(openai.clone()))
    .then_retrieve(qdrant)
    .then_transform_response(OnePerFile { max: 8 })
    .then_answer(answers::Simple::from_client(openai));
```

Source: [`snippets/src/querying/reranking.rs#L48-L52`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/reranking.rs#L48-L52)

For a small change, a closure works too. It must be synchronous, take a `Query<states::Retrieved>` and return `anyhow::Result<Query<states::Retrieved>>`. It must also be `Clone + Send + Sync`, so capture only values that are themselves `Clone`:

```rust
let pipeline = query::Pipeline::default()
    .then_transform_query(query_transformers::Embed::from_client(openai.clone()))
    .then_retrieve(qdrant)
    .then_transform_response(|mut query: Query<states::Retrieved>| {
        query.documents_mut().truncate(5);
        Ok(query)
    })
    .then_answer(answers::Simple::from_client(openai));
```

Source: [`snippets/src/querying/reranking.rs#L60-L67`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/reranking.rs#L60-L67)

## Gotchas

- In 0.33 the plain `Qdrant` retriever returns `Debug`-quoted content, so `Rerank` scores the quoted text. See the gotcha in [Retrieval and search strategies](/querying/retrieval-and-search-strategies/#gotchas) for the docs-qa workaround.
- `Rerank` changes neither `current()` nor the transformation history. The `Retrieved` events in `history()` hold copies of the documents as the retriever returned them. Comparing them with `documents()` shows what was dropped only if no wrapper edited the documents after retrieval and the query retrieved once.
- The `Ragas` evaluator records contexts before `Rerank` runs, so its dataset holds the retrieved candidates, not the kept documents; see [The dataset](/querying/evaluation/#the-dataset). A custom evaluator can read the reranked documents at its second call, after the answer.
- To use another rerank model, build a `TextRerank` yourself, wrap it in `Arc::new(tokio::sync::Mutex::new(..))` and pass it to `model`. That needs direct `fastembed` and `tokio` dependencies, because Swiftide does not re-export `fastembed`.
- Clones of a `Rerank` share one model behind a Tokio mutex. Scoring is synchronous CPU work, so concurrent queries, such as those from `query_all`, take turns at the rerank step.
- If retrieval returns `top_k` documents or fewer, `Rerank` only reorders them. Retrieve more candidates than you keep.

## Reference

- [`Rerank`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.Rerank.html)
- [`RerankBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.RerankBuilder.html)
- [`response_transformers::Summary`](https://docs.rs/swiftide/0.34/swiftide/query/response_transformers/struct.Summary.html)
- [`TransformResponse`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.TransformResponse.html)
- [`Query`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Query.html)
- [`Document`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Document.html)
- [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html)
- [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html)
- The [docs-qa showcase](/showcases/docs-qa/) and the [FastEmbed integration](/integrations/fastembed/)

## Next

[Generating answers](/querying/answering/)