Caching

Add a node cache to your indexing pipeline and a second run skips every file whose path and content have not changed, before any large language model (LLM) call or embedding is made.

Redb cache

The ask-your-codebase showcase keeps its node cache in an embedded Redb file next to the code it indexes:

/// Remembers which files were fully indexed, so unchanged files are skipped next time.
pub fn node_cache(&self) -> Result<Redb> {
    std::fs::create_dir_all(&self.data_dir)?;

    Redb::builder()
        .database_path(self.data_dir.join("cache.redb"))
        .table_name("indexed_files")
        .build()
}

/// Embedded vector store with one vector per chunk and the source path next to it.
pub fn vector_store(&self) -> Result<LanceDB> {
    let uri = self.lancedb_dir();

    LanceDB::builder()
        .uri(uri.to_str().context("Index path is not valid UTF-8")?)
        .table_name("code_chunks")
        .vector_size(embedding::DIMENSIONS)
        .with_vector(EmbeddedField::Combined)
        .with_metadata(PATH_FIELD)
        .build()
}

Source: examples/ask-your-codebase/src/codebase.rs#L56-L77

Redb::builder() takes a database_path and creates the file if it does not exist. It does not create missing parent directories, so the showcase calls create_dir_all first. You can pass an open redb::Database with database instead of a path. table_name defaults to swiftide. The second function builds the LanceDB storage, covered on Storage.

The pipeline adds the cache with filter_cached:

let stats = indexing::Pipeline::from_loader(
    FileLoader::new(codebase.root()).with_extensions(&languages.extensions()),
)
.filter(test_code::not_test_files(codebase.clone()))
// Drop unit tests before the cache, which keys on path and content. Changing the content
// after it would make the cache miss on every run.
.then(test_code::without_unit_tests)
.then(files_found.count())
// Skip files whose path and content are unchanged since the last successful run
.filter_cached(codebase.node_cache()?)
.then(files_changed.count())
.then(record_relative_path(codebase.clone()))
// Split along syntax boundaries (functions, impls, ...) with tree-sitter
.then_chunk(ChunkByLanguage::try_new(languages, CHUNK_SIZE)?)
// Ask the LLM which questions each chunk answers; stored as metadata and embedded
.then(
    MetadataQACode::from_client(llm)
        .prompt_template(QUESTIONS_PROMPT)
        .num_questions(QUESTIONS_PER_CHUNK)
        .concurrency(concurrent_requests)
        .build()?,
)
// Embed chunk + metadata locally, in batches
.then_in_batch(Embed::new(embedding::local_model()?).with_batch_size(embedding::BATCH_SIZE))
.then_store_with(codebase.vector_store()?)
.run()
.await?;

Source: examples/ask-your-codebase/src/index.rs#L89-L115

filter_cached drops every node the cache knows. Every other node continues to the chunker, the LLM and the embedder.

The first run indexes the swiftide-agents crate from scratch, with claude-haiku-5-5 writing the questions and FastEmbed computing the embeddings, as described in Output:

$ ./target/debug/ask-your-codebase index swiftide-agents --model claude-haiku-5-5
Indexing Rust files in swiftide-agents ...
Indexed 34 chunks from 14 new or changed files (14 files found, 0 unchanged) in 91.9s

The second run, with no file changed:

$ ./target/debug/ask-your-codebase index swiftide-agents --model claude-haiku-5-5
Indexing Rust files in swiftide-agents ...
Indexed 0 chunks from 0 new or changed files (14 files found, 14 unchanged) in 0.6s

The cache skips all 14 files, so no prompts are sent and nothing is embedded.

Cache keys

The cache key is a node id: a UUID (universally unique identifier) computed from the node’s path and chunk, not its metadata. The built-in chunkers set parent_id on every chunk to the id of the node that entered the pipeline, the first ancestor, even across several chunkers. Caches use parent_id when it is set. One key therefore covers a file and all of its chunks.

Each backend puts a prefix in front of the id:

Backend	Key format	Prefix setting
Redb	`<prefix>.<uuid>`	`cache_key_prefix`, empty by default
Redis	`<prefix>:<uuid>`	second argument of `Redis::try_from_url`, or `cache_key_prefix` on `Redis::builder()`, empty by default
DuckDB	`<prefix>.<uuid>`	`cache_key_prefix`, empty by default

The pipeline records the id a node has when it passes filter_cached. At the end it marks the parent_id of each finished node, or its own id when parent_id is not set. The file is cached only when the two ids match. Three things break the match silently.

A step after filter_cached that rewrites chunk or path in place changes the node’s id. The file is then never marked and is processed again on every run, with or without a chunker. Keep content-changing steps in front of filter_cached, like the showcase does with test_code::without_unit_tests. Steps that only add metadata are safe after it.

A custom chunker must keep the parent. The built-in chunkers build chunks with Node::into_chunks, which sets parent_id. A closure or ChunkerTransformer that creates fresh nodes with TextNode::new or Node::builder leaves parent_id empty, so the file is never marked. Derive chunks with into_chunks or Node::build_from_other.

The path is part of the id. The same file loaded through ./src and through an absolute path has two keys. The showcase canonicalizes the codebase root for this reason.

When nodes are cached

Since 0.33, run marks a file in the cache only after the stream has completed and every chunk of that file reached the end. A file with a failed chunk is left out and retried on the next run.

Two consequences follow:

If run returns an error, nothing from that run is cached. The same holds when the process stops before run returns.
A source whose chunker produces zero chunks never reaches the end of the stream. It is never cached, and every step up to the chunker runs for it again on each run.

To keep progress when some nodes fail, log and drop the errors so that run completes. This snippet needs the redis, qdrant and openai feature flags:

// Change the prefix, for example to "docs-v2", to index every file again
let cache = Redis::try_from_url(redis_url, "docs-v1")?;

Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"]))
    .filter_cached(cache)
    .then_chunk(ChunkMarkdown::from_chunk_range(100..1500))
    .then(MetadataQAText::new(openai.clone()))
    // Log and drop failed chunks so the run completes. Their files stay uncached and are
    // retried next run; every other file is cached.
    .log_errors()
    .filter_errors()
    .then_in_batch(Embed::new(openai).with_batch_size(32))
    .then_store_with(
        Qdrant::builder()
            .vector_size(1536)
            .collection_name("docs")
            .build()?,
    )
    .run()
    .await?;

Source: snippets/src/indexing/caching.rs#L15-L34

The file behind a failed chunk stays out of the cache and is retried next run. Every other file is cached. Errors covers where to place log_errors and filter_errors.

Backends

Backend	Feature flag	Node type	Notes
`Redb`	`redb`	`TextNode`	Embedded file, no service to run
`Redis`	`redis`	any chunk type	Shared between machines; also a storage
`Duckdb`	`duckdb`	any chunk type	Cache table `swiftide_cache` by default; also a storage

Turn on the feature flag for the backend you pick:

cargo add swiftide@0.34 --features redb

Redis::try_from_url returns a cache for String chunks. For another chunk type, use Redis::try_build_from_url or Redis::builder().

With Redis, the cache is one line. This example also needs the qdrant, openai and tree-sitter feature flags:

indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["rs"]))
    .filter_cached(Redis::try_from_url(redis_url, "swiftide-examples")?)
    .then(MetadataQACode::new(openai_client.clone()))
    .then_chunk(ChunkCode::try_for_language_and_chunk_size(
        "rust",
        10..2048,
    )?)
    .then_in_batch(Embed::new(openai_client.clone()).with_batch_size(10))
    .then_store_with(
        Qdrant::builder()
            .batch_size(50)
            .vector_size(1536)
            .collection_name("swiftide-examples")
            .build()?,
    )
    .run()
    .await?;

Source: examples/index_codebase.rs#L52-L68

MetadataQACode runs before chunking here. That is safe, because it only adds metadata and the id does not change.

No backend fails the pipeline on a cache error. get logs the error and reports the node as not cached, so an unreachable Redis means every node is processed again.

Custom cache

To write your own cache, implement NodeCache. This one keeps the ids in memory:

use std::{
    collections::HashSet,
    sync::{Arc, Mutex},
};

use async_trait::async_trait;
use swiftide::{indexing::TextNode, traits::NodeCache};
use uuid::Uuid;

/// Remembers indexed sources in memory, for tests or one-off runs.
#[derive(Debug, Clone, Default)]
pub struct InMemoryCache {
    indexed: Arc<Mutex<HashSet<Uuid>>>,
}

/// The same key the pipeline marks: the source id, shared by all chunks of a file.
fn source_id(node: &TextNode) -> Uuid {
    node.parent_id().unwrap_or_else(|| node.id())
}

#[async_trait]
impl NodeCache for InMemoryCache {
    type Input = String;

    async fn get(&self, node: &TextNode) -> bool {
        match self.indexed.lock() {
            Ok(indexed) => indexed.contains(&source_id(node)),
            Err(error) => {
                tracing::error!("node cache lock poisoned: {error}");
                false
            }
        }
    }

    async fn set(&self, node: &TextNode) {
        self.set_by_id(source_id(node)).await;
    }

    async fn set_by_id(&self, id: Uuid) {
        match self.indexed.lock() {
            Ok(mut indexed) => {
                indexed.insert(id);
            }
            Err(error) => tracing::error!("node cache lock poisoned: {error}"),
        }
    }
}

Source: snippets/src/indexing/caching.rs#L41-L87

Implement it with #[async_trait], declare type Input as the chunk type, and make the type Send, Sync, Debug and Clone. get returns bool, and set and set_by_id return nothing, so log errors instead of returning them.

Key on parent_id, falling back to id(), so that get, set and set_by_id agree. The pipeline calls set_by_id with the source id when it marks a file. It is a required method since 0.33, so existing custom caches must add it. clear is optional, and its default implementation panics.

Cache invalidation

NodeCache::clear empties a cache, but each backend does it differently:

Redb deletes its table.
DuckDB drops its cache table and does not recreate it. Every later lookup on the same Duckdb logs an error and misses, so build a new Duckdb before the next run.
Redis sends DEL with the pattern <prefix>*. Redis treats that as a literal key name, so the call removes nothing. With an empty prefix, clear returns an error instead. Change the prefix, for example from docs-v1 to docs-v2 in the partial-progress snippet.

A new prefix works on every backend. Use it to index everything again, for example after you change a prompt or the embedding model.

Clearing the cache does not touch the storage. A changed file’s new chunks get new ids and are stored next to the old ones, which stay in LanceDB until you delete the .ask-your-codebase/ directory.

Reference

Storage