Using CriogenAI in your app

CriogenAI's engine is native code that runs on your server, never in the browser. Your React, Vue or mobile app calls your server; your server runs the agent and keeps the keys.

Where it runs#

In Next.js, keep the engine out of the bundle and use the Node runtime:

// next.config.mjs
export default { serverExternalPackages: ["criogenai"] };

// app/api/agent/route.ts
export const runtime = "nodejs";

Importing criogenai never loads the engine. The first use does; if this system cannot run it, that call throws NativeUnavailableError (code: "native_unavailable") with details.platform, details.arch, details.libc and the package it tried, so one route fails with a clear error instead of the whole API at import.

Health check#

import { ready, kaggle } from "criogenai";

const report = await ready({ model: kaggle("google/gemini-3.5-flash-lite") });
// { ok, engine: { ok, platform, arch, libc }, model: { ok, model, latencyMs, error?, code? } }

Without model it only checks the engine (free). With a model it makes one tiny call (a few tokens), so credentials and the provider are checked too. It never throws; use it in /health. Python: await criogenai.ready(model) or criogenai.ready_sync(model).

Typed (structured) output#

Give the agent a JSON Schema (TypeScript) or a Pydantic model (Python). The engine checks every answer against it; an invalid answer goes back to the model with the problem, up to output_retries times (default 2), and then the call fails with output_invalid whose details hold problem and the raw answer. A successful result has the checked object in result.output.

const agent = new Agent({ model, output: {
  type: "object",
  properties: { triage: { enum: ["routine", "urgent", "emergency"] }, follow_up_days: { type: "integer" } },
  required: ["triage", "follow_up_days"],
} });
const { output } = await agent.run("Mild sore throat for two days, no fever.");

Usage, latency and cost#

Every result has usage (tokens), durationMs, answeredBy (the provider and model that wrote the answer, so a fallback shows) and models (calls and tokens per model). Prices change too often to ship, so you pass yours:

import { estimateCost } from "criogenai";

const usd = estimateCost(result, {
  "openai:gpt-5-mini": { input: 0.25, output: 2 },          // USD per million tokens
  "kaggle:google/gemini-3.5-flash-lite": { input: 0, output: 0 },
});  // null if a model in the run has no price

Python: result.duration_ms, result.answered_by, result.models, criogenai.estimate_cost(result, prices).

Redaction before anything leaves#

const agent = new Agent({ model, tools, redact: (text) => text.replace(/MRN-\d+/g, "[MRN]") });

redact runs on every text before each model call: the user's input, earlier replies and tool results (including strings inside JSON). It runs before all hooks, may be async, and the redacted transcript is what a thread saves. It is your rule, applied everywhere: for a hospital, this is where "no patient identifiers to a hosted model" is enforced. It does not detect identifiers by itself; write (and test) the rules for your data.

Tests without a model#

import { Agent, FakeModel, FakeReply } from "criogenai";

const model = new FakeModel([FakeReply.toolCall("lookup_order", { order_id: "A-1" }), "Delivered."]);
const result = await new Agent({ model, tools: [lookupOrder] }).run("Where is A-1?");

FakeModel answers from the script (strings are text replies), never touches the network, and can fail its first calls (failFirst) to test retries and fallbacks.

Streaming, cancelling and timeouts#

const stream = agent.thread("chat-42").stream("Draft the discharge note.");
for await (const event of stream) {
  if (event.type === "text") send(event.text);           // tokens as they arrive
}
const reply = await stream.reply();                        // the saved result

Every call takes an AbortSignal: agent.run(input, { signal: AbortSignal.timeout(20_000) }) stops the run after 20 s; the user's "stop" button can call controller.abort(). Leaving the for await loop early cancels a stream. A cancelled thread call is still saved, and retry() continues it. Tools have their own timeout (limits.toolTimeoutSecs, default 60 s).

To show the answer in a browser as it is written, send the events as Server-Sent Events from a route handler and read them with fetch in the page: the engine stays on the server, and the browser needs no CriogenAI package. Deploying has the route for FastAPI and Next.js. Send error codes to the browser, not error messages: a message can quote what the model or a tool saw.

Tracing#

import { Telemetry } from "criogenai";

const agent = new Agent({ model, telemetry: Telemetry.otlp("https://otel.example.com/v1/traces") });

Each run is a trace with a span per model call and per tool call, exported over OTLP/HTTP without message content unless you turn content capture on. See Tracing.

Errors#

Every failure has a stable code and a class you can catch (ModelError, LimitExceededError, ValidationError, NativeUnavailableError, ...): the error codes list all of them, with what each means and whether a retry can help.

Next#

• Deploying: FastAPI and Next.js routes, storage per deployment, serverless.
• Encryption, audit, retention for sensitive data.
• Troubleshooting and FAQ.