Streaming

send returns when the turn is over, which can take seconds or minutes with tools. stream runs the same turn and gives you its events as they happen: the answer as the model writes it, each tool call as it starts and ends, and a pause for approval. At the end, reply() gives the saved reply, exactly what send would have returned. Use streaming for chat interfaces and anything a person watches.

stream = agent.thread("user-42").stream("Summarise my last order")
async for event in stream:
    if event.type == "text":
        print(event.text, end="")
    elif event.type == "tool_call_started":
        print(f"\n[{event.tool}…]")
    elif event.type == "approval_needed":
        print("\nwaiting for approval")
reply = await stream.reply()  # saved, like send()
# Leaving the loop early cancels the run; it is still saved, and retry() continues it.

const stream = agent.thread("user-42").stream("Summarise my last order");
for await (const event of stream) {
  if (event.type === "text") process.stdout.write(event.text);
  else if (event.type === "tool_call_started") console.log(`\n[${event.tool}…]`);
  else if (event.type === "approval_needed") console.log("\nwaiting for approval");
}
const reply = await stream.reply(); // saved, like send()
// Breaking out of the loop cancels the run; it is still saved, and retry() continues it.

use criogenai_agent::AgentEvent;

let mut stream = agent.thread("user-42").stream("Summarise my last order");
while let Some(event) = stream.next().await {
    match event {
        AgentEvent::Text { text, .. } => print!("{text}"),
        AgentEvent::ToolCallStarted { tool, .. } => println!("\n[{tool}…]"),
        AgentEvent::ApprovalNeeded(requests) => println!("\nwaiting for approval"),
        _ => {}
    }
}
let reply = stream.finish().await?; // saved, like `send`

Events#

Each event has a type; the other fields depend on it.

• text events are pieces: append them. When the message is complete, a message event carries all of it; prefer that to your own concatenation if you store it.
• Treat other as diagnostics: it has the engine event's name (event, e.g. step_started), its node and step, and never any content. Use tracing for the details.
• In Python, reading a field the event type does not have raises AttributeError; check event.type first.

Streaming the other operations#

Streaming is for threads. A one-off agent.run does not stream: use a thread with a fresh id if you need events for a single task.

When the reader goes away#

Leaving the loop early (break, an exception in your loop body, cancelling the task), calling stream.cancel(), or a browser closing the connection cancels the run instead of leaving it running unseen:

• the model call stops at once, and no tool calls start;
• the run is saved as failed with cancelled, and reply() raises CancelledError;
• retry() (or stream_retry()) continues it later. Nothing half-written is left in the thread.

If you want the turn to finish even when nobody watches, use send in a background task and let the page poll for the result.

To a browser#

Run the stream on your server and forward the events as Server-Sent Events. The browser needs no CriogenAI package: it reads the events with fetch or EventSource. Send only what the page needs (text, tool names, done), and send error codes, not messages. Complete examples for FastAPI and a Next.js route handler are on Deploying.

Errors#

Errors reach you from reply() (and, for some, from the loop): the same codes as send, such as thread_busy, thread_suspended, model_failed and cancelled. A stream started on a suspended thread fails with thread_suspended.

Next#

• Deploying: Server-Sent Events from FastAPI and Next.js.
• Conversations (threads): retry and history.
• Tracing: see every run's steps after the fact.