Human in the loop

An agent can stop before a tool runs, wait for a person to approve or refuse the call, and then continue where it stopped.

The agent types on this page live in swiftide::agents, behind the swiftide-agents feature flag. The example also uses the openai integration and the default macros feature flag:

cargo add swiftide@0.34 --features openai,swiftide-agents

Approval wrapper

// ApprovalRequired is a simple wrapper. You can also implement your own approval
// flows by returning a `ToolOutput::FeedbackRequired` in a tool,
// you can then use `has_received_feedback` and `received_feedback` on the context
// to build your custom workflow.
let guess_with_approval = ApprovalRequired::new(guess_a_number());

Source: examples/agents_with_human_in_the_loop.rs#L60-L64

The comment in the example gets one method name wrong: the method on the context that stores feedback is feedback_received.

let mut agent = agents::Agent::builder()
    .llm(&openai)
    .tools(vec![guess_with_approval])
    // Every message added by the agent will be printed to stdout
    .on_new_message(move |_, msg| {
        println!("{msg}");

        Box::pin(async move { Ok(()) })
    })
    .limit(5)
    .build()?;

Source: examples/agents_with_human_in_the_loop.rs#L68-L78

ApprovalRequired wraps any tool. It keeps the inner tool’s name and spec, so the model sees the same tool. When the model calls it, the wrapper asks the context for feedback on that tool call:

No feedback yet: it returns ToolOutput::FeedbackRequired(None). The agent stops with StopReason::FeedbackRequired { tool_call, payload } and records no tool output, so the tool call stays open.
Approved: it runs the inner tool and returns its output.
Refused: it returns the text This tool call was refused, which the model reads as the tool output.

query returns Ok(()) when the agent stops this way. Read the stop reason to find the call that waits:

// First query the agent, the agent will stop with a reason that feedback is required
agent
    .query("Guess a number between 0 and 100 using the `guess_a_number` tool")
    .await?;

// The agent stopped, lets get the tool call
let Some(StopReason::FeedbackRequired { tool_call, .. }) = agent.stop_reason() else {
    panic!("expected a tool call to approve")
};

// Alternatively, you can also get the stop reason from the agent state
// agent.state().stop_reason().unwrap().feedback_required().unwrap()

// Register that this tool call is ok.
println!("Approving number guessing");
agent
    .context()
    .feedback_received(tool_call, &ToolFeedback::approved())
    .await
    .unwrap();

// Run the agent again and it will pick up where it stopped.
agent.run().await.unwrap();

Source: examples/agents_with_human_in_the_loop.rs#L82-L104

feedback_received on the context stores a ToolFeedback for that tool call. run adds no user message. Before it asks the model for anything, it invokes the tool calls of the last assistant message again, provided no tool output follows that message. This time the wrapper finds the approval and runs the inner tool, and the loop carries on as usual.

Approval loop

The release notes showcase wraps its write_changelog tool as OneEntryPerChange::new(ApprovalRequired::new(WriteChangelog::new(..))) and loops until a maintainer approves a draft or gives up:

/// Asks the agent for release notes and loops through review rounds until they are written or
/// rejected.
pub async fn draft_release_notes(
    agent: &mut Agent,
    task: Prompt,
    reviewer: &mut impl Reviewer,
) -> Result<Outcome> {
    agent.query(task).await?;
    let mut written = false;
    let mut nudged = false;

    loop {
        // `write_changelog` is wrapped in `ApprovalRequired`: the agent stops and waits.
        let tool_call = match agent.stop_reason() {
            Some(StopReason::FeedbackRequired { tool_call, .. }) => tool_call.clone(),
            // The custom `stop` tool carries the structured release notes.
            Some(StopReason::RequestedByTool(_, Some(notes))) if written => {
                let notes = ReleaseNotes::deserialize(notes)
                    .context("the agent stopped with malformed release notes")?;
                return Ok(Outcome::Written { notes: Some(notes) });
            }
            _ if written => return Ok(Outcome::Written { notes: None }),
            // A turn without a tool call, or the step limit.
            Some(StopReason::NoNewMessages) => {
                nudge_once(agent, &mut nudged).await?;
                continue;
            }
            other => bail!("the agent stopped without writing the notes: {other:?}"),
        };

        let proposal: WriteChangelogArgs =
            serde_json::from_str(tool_call.args().unwrap_or_default())?;

        match reviewer.review(&proposal)? {
            Verdict::Approve => {
                agent
                    .context()
                    .feedback_received(&tool_call, &ToolFeedback::approved())
                    .await?;
                // Resumes the pending tool call, which now runs, and continues the loop.
                agent.run().await?;
                // The write itself can still fail; the agent then hears why and carries on.
                written |= succeeded(agent, &tool_call).await?;
            }
            Verdict::Revise(feedback) => {
                agent
                    .context()
                    .feedback_received(&tool_call, &ToolFeedback::refused())
                    .await?;
                // The refusal is recorded as the tool's output, followed by the feedback.
                agent.query(feedback).await?;
            }
            Verdict::Reject => return Ok(Outcome::Rejected),
        }
    }
}

Source: examples/release-notes-agent/src/session.rs#L41-L96

The arguments of the waiting call are the proposal. The session parses them into the arguments struct that #[derive(Tool)] generates and hands them to a Reviewer, a small trait with a terminal implementation in terminal.rs.

After an approval the write can still fail, so the session checks the history for a ToolOutput::Text answering that call before it counts the notes as written.

Captured approval

This is the approval excerpt of a real run on the Swiftide repository. The model was claude-sonnet-5-5 behind an OpenAI-compatible gateway, and y was piped into stdin, which the reviewer echoes:

  → write_changelog <32 lines>
  write_changelog ⏸ waiting for approval

── proposed: 32 lines into a new file, RELEASE_NOTES.md
Write these release notes? [y/N] y
  → write_changelog <32 lines>
  write_changelog ✓ wrote RELEASE_NOTES.md
  → stop (version 0.34.0: highlights 2, breaking 1, fixes 1)
  stop ■ done

The → lines come from a before_tool hook and the result lines from an after_tool hook. ⏸ waiting for approval is the hook seeing ToolOutput::FeedbackRequired. The second → write_changelog is the same tool call, invoked again by run() after the approval.

Refusals and revisions

A refusal is an answer, not an error. ToolFeedback::refused() makes the wrapper return its refusal text, and the agent records it as the tool output. query(feedback) queues the person’s message until that output is recorded, so the history reads: tool call, refusal, feedback. The model then drafts again and calls write_changelog with a new tool call, which starts a new approval round.

Neither approved() nor refused() takes an argument. To attach a payload, chain .with_payload(serde_json::Value) after the constructor. ApprovalRequired ignores it; your own tools can read it.

Draft checks

OneEntryPerChange in the showcase’s tools.rs wraps the approval tool. It parses the draft and returns ToolOutput::fail when one pull request or commit appears in more than one entry; otherwise it calls the wrapped tool. The model reads the failure and fixes the draft, and the maintainer only sees drafts that pass. A ToolOutput::fail does not count toward the tool retry limit (Tool errors and retries). Put checks outside the approval wrapper, so they run before anyone is asked.

Custom feedback tools

A tool can ask for feedback itself by returning ToolOutput::feedback_required(Some(payload)). The payload reaches the caller as the payload of StopReason::FeedbackRequired. The tool needs the ToolCall to look up feedback, so it implements Tool by hand; #[swiftide::tool] functions and #[derive(Tool)] structs only receive the context and their arguments.

#[derive(Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
struct RefundArgs {
    /// The order to refund.
    order_id: String,
    /// The amount to refund, in cents.
    amount_cents: u64,
}

#[derive(Clone)]
struct IssueRefund;

#[swiftide::reexports::async_trait::async_trait]
impl Tool for IssueRefund {
    async fn invoke(
        &self,
        context: &dyn AgentContext,
        tool_call: &ToolCall,
    ) -> Result<ToolOutput, ToolError> {
        let args = tool_call
            .args()
            .ok_or_else(|| ToolError::missing_arguments("order_id, amount_cents"))?;
        let refund: RefundArgs = serde_json::from_str(args)?;

        match context.has_received_feedback(tool_call).await {
            // No decision yet: stop the agent and show the reviewer what it asks for.
            None => Ok(ToolOutput::feedback_required(Some(json!({
                "order_id": refund.order_id,
                "amount_cents": refund.amount_cents,
            })))),
            // The reviewer can lower the amount in the approval payload.
            Some(ToolFeedback::Approved { payload }) => {
                let amount_cents = payload
                    .as_ref()
                    .and_then(|payload| payload["amount_cents"].as_u64())
                    .unwrap_or(refund.amount_cents);
                send_refund(&refund.order_id, amount_cents).await?;
                Ok(ToolOutput::text(format!(
                    "Refunded {amount_cents} cents for order {}",
                    refund.order_id
                )))
            }
            Some(ToolFeedback::Refused { payload }) => {
                let reason = payload
                    .as_ref()
                    .and_then(|payload| payload["reason"].as_str())
                    .unwrap_or("no reason given");
                Ok(ToolOutput::text(format!(
                    "A person refused the refund: {reason}"
                )))
            }
        }
    }

    fn name(&self) -> Cow<'_, str> {
        Cow::Borrowed("issue_refund")
    }

    fn tool_spec(&self) -> ToolSpec {
        ToolSpec::builder()
            .name("issue_refund")
            .description("Refund (part of) an order. A person approves every refund.")
            .parameters_schema(schema_for!(RefundArgs))
            .build()
            .expect("the refund arguments form a strict schema")
    }
}

Source: snippets/src/agents/human_in_the_loop.rs#L20-L86

DefaultContext removes the feedback when has_received_feedback returns it, so each decision applies to one invocation. The caller can change what the tool does through the payload of the approval:

if let Some(StopReason::FeedbackRequired { tool_call, payload }) = agent.stop_reason() {
    println!("approve {payload:?}?");
    let tool_call = tool_call.clone();
    let feedback = ToolFeedback::approved().with_payload(json!({ "amount_cents": 2_500 }));

    agent
        .context()
        .feedback_received(&tool_call, &feedback)
        .await?;
    agent.run().await?;
}

Source: snippets/src/agents/human_in_the_loop.rs#L92-L102

Gotchas

Have the model call an approval tool on its own. run and query only resume open tool calls when no tool output follows the assistant message. If the same completion also called another tool, that tool’s output is recorded and the approved call never runs. The showcase prompt says “Call write_changelog with the Markdown, on its own.”
The stop reason names only the first tool call that asked for feedback in a completion.
Calling run() or query() before giving feedback invokes the wrapper again, which asks again, and the agent returns AgentError::UnfulfilledToolCalls. A message passed to that query stays queued, and the next successful run() or query() adds it to the history after the tool output. Retry with run() so the message does not appear twice.
Hooks see the resumed call twice (Lifecycle hooks).
Feedback lives in the context, not the message history. To approve in another process, see Context and history.

Reference

ApprovalRequired
ToolFeedback
StopReason
Agent (query, run, stop_reason, context)
AgentContext (feedback_received, has_received_feedback)
DefaultContext (with_tool_feedback)
ToolOutput
Tool, ToolCall, ToolSpec
AgentError

Streaming