Skip to content
swiftide

Glossary

You get a short definition of each core term in the Swiftide docs, grouped by section and linked to the page that explains it.

Agents

Section titled “Agents”
  • Agent: a loop around a large language model (LLM). Each turn, it sends the message history to the model and runs the tool calls it returns. Build one with Agent::builder().
  • Tool: a function the model can call by name with JSON arguments. It implements the Tool trait. The #[tool] macro and the Tool derive write it for you.
  • Hook: a closure the agent runs at a fixed point in its loop, such as before_all, before_tool, after_each or on_stop. Most hooks receive &mut Agent.
  • Tool executor: where tools run commands and read or write files. It implements ToolExecutor. The default, LocalExecutor, runs commands on your machine.
  • Context: the state an agent runs in. It implements AgentContext and holds the message history and the tool executor. The default is DefaultContext.
  • Message history: the storage that keeps a context’s chat messages. It implements MessageHistory. The default keeps messages in memory. Redis from the redis integration keeps them between runs.
  • Stop reason: why an agent stopped, for example because a tool asked it to or it failed. Read it with stop_reason(). Its type is StopReason.

Tasks

Section titled “Tasks”
  • Task: a graph of typed steps with one input and one output type. Running a Task returns a TaskRunOutcome: the output, or paused.
  • Task node: one typed step in a task, with an input, an output and an error type. It implements TaskNode. Task::register_node_fn turns a closure into a node.
  • Transition: the value that says where a task goes after a node. Register it on that node with Task::register_transition and a function from the node’s output to a Transition. In that function, next.transitions_with(value) moves to node next. Pass task.transitions_to_finish() as the function to end the task.
  • Fan-out: a transition that starts several branches at once. Transition::fan_out starts it, and adds a branch, and join_with(join.join()) names the join that waits for every branch. Until then it is a FanOutTransition, which cannot run.
  • Join: a node that receives the outputs of every fan-out branch together as a JoinInput. NodeId::join gives the join target. Each branch registers its own transition to that target.

Indexing

Section titled “Indexing”
  • Indexing pipeline: a stream that loads data as nodes, transforms them and stores them. Start one with Pipeline::from_loader from swiftide::indexing and run it with run.
  • Node: one unit of data in an indexing pipeline: a path, a chunk, metadata and embeddings. Node is generic over its chunk type. TextNode is a node whose chunk is a String.
  • Chunk: the content a node carries, in its chunk field. Any type that is Clone, Send, Sync, Debug, 'static and readable as bytes implements the Chunk trait automatically.
  • Loader: the source at the start of an indexing pipeline. It implements Loader and turns files, pages or messages into nodes. FileLoader reads files from disk.
  • Transformer: a step that maps one node to one node, for example to add metadata. It implements Transformer. Add it with then.
  • Batch transformer: a step that takes many nodes in one call. It implements BatchableTransformer. Add it with then_in_batch.
  • Chunker: a step that splits one node into many. It implements ChunkerTransformer, and ChunkMarkdown is one. Add it with then_chunk.
  • Embedder: a model that turns text into vectors. Dense models implement EmbeddingModel and Embed calls them. Sparse models implement SparseEmbeddingModel and SparseEmbed calls them.
  • Persist: the trait a pipeline calls to store nodes. Add a storage with then_store_with.
  • Storage: the backend that implements Persist, such as Qdrant, PgVector, Redis or LanceDB. Their flags are qdrant, pgvector, redis and lancedb.
  • Node cache: a record of the nodes a pipeline already processed. It implements NodeCache. filter_cached drops nodes found in the cache. A node enters it only after passing the whole pipeline.

Querying

Section titled “Querying”
  • Query pipeline: a pipeline that turns a question into an answer in four stages: query transformers (then_transform_query), a retriever (then_retrieve), response transformers (then_transform_response) and an answer generator (then_answer). Start one with Pipeline::from_search_strategy from swiftide::query and ask with query.
  • Retriever: a store that fetches documents with a search strategy. It implements Retrieve for that strategy. Qdrant, PgVector, LanceDB (lancedb flag) and DuckDB (duckdb flag) are retrievers.
  • Search strategy: settings for how a retriever searches, such as result count and filter. It implements SearchStrategy. Built-in ones are SimilaritySingleEmbedding, HybridSearch and CustomStrategy.
  • Response transformer: a step between retrieval and the answer. It implements TransformResponse. Summary replaces the response with a summary of the documents. The FastEmbed Rerank (fastembed flag) reorders the documents.
  • Answer generator: the last step, which writes the answer from the documents. It implements Answer. Simple from swiftide::query::answers asks an LLM.
  • Evaluator: a component that sees the retrieved documents and the final answer of each query. It implements EvaluateQuery. Add it with evaluate_with. Ragas exports a dataset for the Ragas framework.

Setup

Section titled “Setup”
  • Feature flag: a Cargo feature of swiftide that compiles in an optional part. Only macros is on by default. Agents need swiftide-agents, tasks need swiftide-tasks, and each integration needs its own flag.
  • Integration: a module under swiftide::integrations that wraps an outside library or service. Examples are providers (openai), stores (qdrant), loaders (scraping) and parsers (treesitter). Each has its own feature flag.

Reference

Section titled “Reference”

Next: Contributing and community