Prompting and Embedding

Our metadata transformers are generic over the SimplePrompt trait. This enables different models to be used for different usecases. Similarly, the embedding transformer is generic over the EmbeddingModel trait.

The SimplePrompt trait

Which is defined as follows:

pub trait SimplePrompt: Debug + Send + Sync {
    async fn prompt(&self, prompt: &str) -> Result<String>;
}

Or in human language: “Given a Prompt, give me a response”.

The EmbeddingModel trait

Which is defined as follows:

pub trait EmbeddingModel: Send + Sync {
    async fn embed(&self, input: Vec<String>) -> Result<Embeddings>;
}

Or in human language: “Given a list of things to Embed, give me embeddings”. The embedding transformer will link back the embeddings to the original nodes by order.

Built in inference and embedding models

Name	Description	Feature Flag
OpenAI	Implements both SimplePrompt and Embed via async_openai	openai
FastEmbed	Implements Embed via fastembed-rs	fastembed
Titan (Bedrock)	AWS Titan via Bedrock	aws-bedrock
Anthropic models (Bedrock)	Recen Anthropic models via Bedrock	aws-bedrock