Generating answers

The last step of a query pipeline sends the question and the retrieved documents to a large language model (LLM). Here you set that step up so every answer names the file it came from, the way the ask-your-codebase showcase does.

answers::Simple needs no feature flag. The snippets below use the OpenAI client and Qdrant, which sit behind the openai and qdrant feature flags:

cargo add swiftide@0.34 --features openai,qdrant

let search = SimilaritySingleEmbedding::default().with_top_k(top_k);

let pipeline = query::Pipeline::from_search_strategy(search)
    // Rephrase the question into several, to widen what the search can match
    .then_transform_query(
        GenerateSubquestions::builder()
            .client(llm.clone())
            .prompt_template(SUBQUESTIONS_PROMPT.into())
            .build()?,
    )
    // Embed with the same local model that embedded the chunks
    .then_transform_query(Embed::from_client(embedding::local_model()?))
    .then_retrieve(codebase.vector_store()?)
    // Answer the original question from the retrieved code, each chunk under its file path
    .then_answer(
        answers::Simple::builder()
            .client(llm)
            .prompt_template(ANSWER_PROMPT.into())
            .document_template(CHUNK_TEMPLATE)
            .build()?,
    );

let answered = pipeline.query(question).await?;

Source: examples/ask-your-codebase/src/query.rs#L36-L58

The answer generator

then_answer takes an answer generator and moves the query from Retrieved to Answered. Swiftide ships one: answers::Simple. It renders a prompt, sends it through any client that implements SimplePrompt, and stores the reply as the answer.

answers::Simple::from_client(llm) builds it with the default prompt. The builder lets you replace two templates:

prompt_template is the whole prompt.
document_template renders each retrieved document before they are joined. It is optional.

Without a custom prompt, the model receives this text, shown here with What is love? as the question and My context as the documents:

Answer the following question based on the context provided:
What is love?

## Constraints
* Do not include any information that is not in the provided context.
* If the question cannot be answered by the provided context, state that it cannot be answered.
* Answer the question completely and format it as markdown.

## Context

---
My context
---

The prompt template sees four variables:

question: the question the user asked, from Query::original.
original: the same text as question.
current: the current query, from Query::current.
documents: the retrieved documents as one string, or the text of a response transformer that changed current (see below).

Prompts are Tera templates; Prompts and templates covers the syntax. original and current need 0.33 or later; see Upgrading.

Source citations

The answer can only name a file if the prompt contains the file name. The document template puts it there.

/// Widens the question into the follow-up questions its answer involves.
///
/// They are phrased like the questions generated for each chunk while indexing, so the two
/// embed close together.
const SUBQUESTIONS_PROMPT: &str = r"
Your job is to help a search find the code that answers a developer's question about a codebase.

Question: {{ question }}

Write {{ num_questions }} short follow-up questions that the code answering this question would also
answer. Think about the mechanisms involved: the main loop, the types and enum variants, the
conditions that are checked, the functions that are called, and the configuration that changes
the outcome.

## Constraints
* Phrase them the way a developer would ask about this codebase, e.g. 'When does the agent stop?'.
* Do not guess names of types or functions.
* Respond with the original question and the follow-up questions only, one per line, prefixed
  with '- '.
";

/// Renders a retrieved chunk under its file path, so the answer can cite it.
///
/// `metadata.path` is the `PATH_FIELD` metadata; a test keeps the two in step.
const CHUNK_TEMPLATE: &str = r"### {{ metadata.path }}

```
{{ content }}
```
";

/// Answers from the retrieved code and cites files.
const ANSWER_PROMPT: &str = r"
Answer a developer's question about a codebase, using snippets of its source code.

Question: {{ question }}

## Constraints
* Only use the snippets below. If they do not answer the question, say briefly what is missing.
* Be concrete: name the types, functions and variants involved and explain when they apply.
* Describe behaviour: what calls what, in which order, and under which conditions.
* Cite the file each statement comes from in backticks, e.g. (`src/lib.rs`).
* Use concise markdown. Add a short code example only if it shows how to use the API.

## Snippets

{{ documents }}
";

Source: examples/ask-your-codebase/src/query.rs#L65-L112

CHUNK_TEMPLATE renders one document. Its template context is the serialized Document, so content is the chunk text and metadata holds the metadata fields your storage kept. ask-your-codebase writes more metadata while indexing, but its LanceDB table keeps only path, so {{ metadata.path }} becomes a heading above each chunk. answers::Simple joins the rendered documents with --- lines and inserts the result as {{ documents }}.

ANSWER_PROMPT then asks for a citation after each statement. With claude-sonnet-5-5, the answers follow it:

$ ./target/debug/ask-your-codebase ask --model claude-sonnet-5-5 --path swiftide-agents "How does an agent decide to stop?"
# How an agent decides to stop

An agent stops when something calls `Agent::stop` (or `stop_with_error`), which moves the state to `State::Stopped(StopReason)`. The main loop in `run_agent` checks that state after each step. `Agent::stop` is a no-op if the agent is already stopped (`src/agent.rs`).

## The stop mechanism

- `Agent::stop(reason)` fires the `OnStop` hooks with the reason, then sets `self.state = State::Stopped(reason)`. It only takes effect at the end of the current loop iteration (`src/agent.rs`).
...

Sources:
  - src/agent.rs
  - src/tools/control.rs
  - src/lib.rs
  ...

The Sources: list does not come from the model. The answered query still holds the documents it was answered from, so the showcase reads their paths back:

impl From<Query<states::Answered>> for Answer {
    /// Keeps the distinct file paths of the retrieved chunks, most relevant first.
    fn from(query: Query<states::Answered>) -> Self {
        let mut sources: Vec<String> = Vec::new();

        for document in query.documents() {
            let Some(path) = document.metadata().get(PATH_FIELD).and_then(|v| v.as_str()) else {
                continue;
            };
            if !sources.iter().any(|source| source == path) {
                sources.push(path.to_owned());
            }
        }

        Self {
            text: query.answer().to_owned(),
            sources,
        }
    }
}

Source: examples/ask-your-codebase/src/query.rs#L123-L142

answer returns the text and documents returns the documents the answer was built from. A response transformer can reorder or trim that list first: the docs-qa showcase reranks its documents, so its numbered list of page and section titles comes best first.

Original and current question

A query transformer such as GenerateSubquestions rewrites the current query. question in the prompt stays the user’s question, so the model answers what was asked, not the list of subquestions. Use current when the model should also see what the search looked for:

let answerer = answers::Simple::builder()
    .client(openai)
    .prompt_template(Prompt::from(
        "Answer this question: {{ original }}\n\
         The search looked for: {{ current }}\n\
         Cite the file of every statement in backticks.\n\n\
         {{ documents }}",
    ))
    .document_template("File: {{ metadata.path }}\n{{ content }}")
    .build()?;

Source: snippets/src/querying/answering.rs#L14-L23

answers::Simple decides what goes into documents like this:

If a response transformer changed current after retrieval, documents is that text and the document template is skipped. response_transformers::Summary works this way; see Reranking and response transformers.
Otherwise each document is rendered with the document template, or as its plain content without one.

The check compares current with the query recorded by the Retrieved event, so a rewritten query from before retrieval does not count as a change. A reranker that only reorders documents leaves current alone, so the document template still applies. Transforming queries explains the transformation history.

Custom answer generators

Implement the Answer trait to control the last step yourself. This one skips the LLM call when the retriever found nothing:

/// Answers with the LLM only when the retriever found something.
#[derive(Clone)]
pub struct AnswerWhenFound {
    inner: answers::Simple,
}

#[async_trait]
impl Answer for AnswerWhenFound {
    async fn answer(
        &self,
        query: Query<states::Retrieved>,
    ) -> anyhow::Result<Query<states::Answered>> {
        if query.documents().is_empty() {
            return Ok(query.answered("No matching documents were found."));
        }
        self.inner.answer(query).await
    }
}

Source: snippets/src/querying/answering.rs#L30-L47

Query::answered stores the text and moves the query to Answered. The trait requires Clone, so derive it. Every other case goes to the wrapped answers::Simple. The generator goes where answers::Simple went:

let pipeline = query::Pipeline::default()
    .then_transform_query(query_transformers::Embed::from_client(openai.clone()))
    .then_retrieve(qdrant)
    .then_answer(AnswerWhenFound {
        inner: answers::Simple::from_client(openai),
    });

let answered = pipeline.query("How do I configure retries?").await?;
println!("{}", answered.answer());

Source: snippets/src/querying/answering.rs#L53-L61

then_answer also accepts a synchronous Fn(Query<states::Retrieved>) -> Result<Query<states::Answered>> + Send + Sync + Clone closure, for example a fixed answer in a test. A closure that captures a value which is not Clone does not compile.

Gotchas

prompt_template takes a Prompt, so write .into() or Prompt::from. document_template converts for you, so a &'static str works directly.
A document template that reads a missing metadata key fails to render, and the query returns that error. The docs-qa templates guard each key with Tera’s default filter, as in {{ metadata.Title | default(value="") }}.
After answering, current holds the answer; see Transforming queries for reading the rewritten query from history.
The model can only cite files among the retrieved documents. A chunk outside the top results is invisible to it. Raise top_k for broad questions; see Retrieval and search strategies.
answers::Simple makes one LLM call and returns its error. Wrap the client in LanguageModelWithBackOff to retry transient failures, as docs-qa does; see Errors, backoff and usage.

Reference

Evaluating with Ragas