Python API: Models

Providers and models: OpenAI, Anthropic, Gemini, Azure, Bedrock, Vertex AI, local servers and more. Generated from the docstrings and type hints of criogenai 0.1.7. See also: Python API overview, the same module in TypeScript.

Models: hosted providers, cloud platforms, local servers, and FakeModel for tests.

Pass a model object or a "provider:model" string wherever a model is taken (Agent, Gateway, fallbacks). A string takes the provider's key from its usual environment variable: "openai:gpt-5-mini" from OPENAI_API_KEY, "deepseek:deepseek-chat" from DEEPSEEK_API_KEY (criogenai.env.key_name gives the variable). The providers a string may name: openai, anthropic, gemini, openrouter, deepseek, mistral, xai, groq, cerebras, together, fireworks, qwen, moonshot, zai, ollama, lmstudio, kaggle, azure, bedrock and vertex.

The named providers' classes (OpenAI, Anthropic, Gemini, ...) take (model, api_key=None, base_url=None): api_key defaults to the provider's environment variable, and base_url replaces the provider's URL (a regional endpoint, a proxy). Using a named provider without a key (and without base_url) raises ValidationError (invalid_value; also a ValueError). repr() of a model never shows a key.

from criogenai import (
    OpenAI,
    Anthropic,
    Gemini,
    FakeModel,
    OpenAICompatible,
    OpenRouter,
    DeepSeek,
    Mistral,
    XAI,
    Groq,
    Cerebras,
    Together,
    Fireworks,
    Qwen,
    Moonshot,
    ZAI,
    Kaggle,
    Vertex,
    Bedrock,
    Azure,
    Ollama,
    LMStudio,
    AnthropicCompatible,
    FakeReply,
    Embedder,
)

Contents#

OpenAI#

class OpenAI(_Model)

OpenAI's API, e.g. OpenAI("gpt-5-mini") or "openai:gpt-5-mini".

Constructor: OpenAI(model, api_key=None, base_url=None). The key defaults to OPENAI_API_KEY; base_url replaces OpenAI's URL.

Example

from criogenai import Agent, OpenAI

agent = Agent(OpenAI("gpt-5-mini"), instructions="Answer briefly.")
# the same, with the key from OPENAI_API_KEY:
agent = Agent("openai:gpt-5-mini", instructions="Answer briefly.")

Anthropic#

class Anthropic(_Model)

Anthropic's Messages API, e.g. Anthropic("claude-sonnet-5-5") or "anthropic:claude-sonnet-5-5".

Constructor: Anthropic(model, api_key=None, base_url=None); the key defaults to ANTHROPIC_API_KEY.

Gemini#

class Gemini(_Model)

Google Gemini through its OpenAI-compatible endpoint, e.g. Gemini("gemini-2.5-flash") or "gemini:gemini-2.5-flash".

Constructor: Gemini(model, api_key=None, base_url=None); the key defaults to GEMINI_API_KEY. For Gemini on Google Cloud, see Vertex.

FakeModel#

class FakeModel:
    def __init__(
        self,
        replies: Sequence[Union[str, FakeReply]] = (),
        delay: float = 0.0,
        fail_first: int = 0,
        by_turn: bool = False,
        name: Optional[str] = None,
    )

A simulated model for tests; it never makes a network call.

It answers with the scripted replies in order. When they are used up, it echoes the last user message ("[fake-llm] reply to: ...").

Parameters

• replies (Sequence[Union[str, FakeReply]], default ()): The answers, in order: FakeReply objects, or strings for text answers.
• delay (float, default 0.0): How long each answer takes, in seconds.
• fail_first (int, default 0): The first n calls fail with rate_limited, a transient error that the agent retries; use it to test retries and fallbacks.
• by_turn (bool, default False): Choose each reply by the model call's place in the run (the first reply for a run's first model call, the second for its second; the last repeats) instead of using replies up. One model then serves many runs, also at the same time.
• name (Optional[str], default None): The model name in results (answered_by["model"]; provider "fake"), to tell two fake models apart. Default "fake".

Example

from criogenai import Agent, FakeModel, FakeReply, tool

@tool
def weather(city: str) -> str:
    "Current weather for a city."
    return f"Sunny in {city}"

model = FakeModel([FakeReply.tool_call("weather", {"city": "Paris"}), "It is sunny."])
result = Agent(model, tools=[weather]).run_sync("Weather in Paris?")
assert result.text == "It is sunny." and result.tool_calls == 1

FakeModel.spec#

def spec() -> Dict[str, Any]

The JSON spec the engine builds the model from (internal).

OpenAICompatible#

class OpenAICompatible(_Model):
    def __init__(
        self,
        model: str,
        base_url: str,
        api_key: Optional[str] = None,
        name: str = 'custom',
        allow_insecure_http: bool = False,
    )

Any server that speaks OpenAI's Chat Completions API: vLLM, a gateway, or a provider without a class here.

Parameters

• model (str): The model name the server expects.
• base_url (str): The server's API URL, usually ending in /v1.
• api_key (Optional[str], default None): The key, if the server needs one. No environment variable is read.
• name (str, default 'custom'): The provider name in results and telemetry (answered_by["provider"]).
• allow_insecure_http (bool, default False): Send the key over plain http:// to another machine (a vLLM server on your network). Refused by default: a key sent over http is readable by anyone on the path. Servers on this machine are always allowed.

Example

from criogenai import Agent, OpenAICompatible

model = OpenAICompatible("my-model", base_url="http://localhost:8000/v1")
agent = Agent(model)

OpenAICompatible.spec#

def spec() -> Dict[str, Any]

The JSON spec the engine builds the model from (internal).

OpenRouter#

class OpenRouter(_Model)

OpenRouter: many providers' models behind one key, e.g. OpenRouter("meta-llama/llama-3.3-70b-instruct").

Constructor: OpenRouter(model, api_key=None, base_url=None); the key defaults to OPENROUTER_API_KEY.

DeepSeek#

class DeepSeek(_Model)

DeepSeek, e.g. DeepSeek("deepseek-chat").

Constructor: DeepSeek(model, api_key=None, base_url=None); the key defaults to DEEPSEEK_API_KEY.

Mistral#

class Mistral(_Model)

Mistral AI, e.g. Mistral("mistral-large-latest").

Constructor: Mistral(model, api_key=None, base_url=None); the key defaults to MISTRAL_API_KEY.

XAI#

class XAI(_Model)

xAI's Grok models.

Constructor: XAI(model, api_key=None, base_url=None); the key defaults to XAI_API_KEY.

Groq#

class Groq(_Model)

Groq's inference for open models, e.g. Groq("llama-3.3-70b-versatile").

Constructor: Groq(model, api_key=None, base_url=None); the key defaults to GROQ_API_KEY.

Cerebras#

class Cerebras(_Model)

Cerebras's inference for open models.

Constructor: Cerebras(model, api_key=None, base_url=None); the key defaults to CEREBRAS_API_KEY.

Together#

class Together(_Model)

Together AI's open models.

Constructor: Together(model, api_key=None, base_url=None); the key defaults to TOGETHER_API_KEY.

Fireworks#

class Fireworks(_Model)

Fireworks AI's open models.

Constructor: Fireworks(model, api_key=None, base_url=None); the key defaults to FIREWORKS_API_KEY.

Qwen#

class Qwen(_Model)

Qwen on Alibaba Cloud Model Studio, international endpoint.

Constructor: Qwen(model, api_key=None, base_url=None); the key defaults to DASHSCOPE_API_KEY. For the China endpoint pass base_url="https://dashscope.aliyuncs.com/compatible-mode/v1".

Moonshot#

class Moonshot(_Model)

Moonshot AI's Kimi models.

Constructor: Moonshot(model, api_key=None, base_url=None); the key defaults to MOONSHOT_API_KEY. For China keys pass base_url="https://api.moonshot.cn/v1".

ZAI#

class ZAI(_Model)

Z.ai's GLM models.

Constructor: ZAI(model, api_key=None, base_url=None); the key defaults to ZAI_API_KEY. For China keys pass base_url="https://open.bigmodel.cn/api/paas/v4".

Kaggle#

class Kaggle:
    def __init__(
        self,
        model: str,
        api_token: Optional[str] = None,
        proxy_url: Optional[str] = None,
        proxy_token: Optional[str] = None,
        api_base: Optional[str] = None,
    )

Kaggle's Model Proxy: the hosted models your Kaggle account's model allowance pays for, e.g. Kaggle("google/gemini-3.6-flash") or "kaggle:google/gemini-3.6-flash".

Credentials, in order: proxy_url with proxy_token (a proxy token minted elsewhere, as in a Kaggle notebook); api_token; else the environment: MODEL_PROXY_URL and MODEL_PROXY_API_KEY, KAGGLE_API_TOKEN, or the access_token file in KAGGLE_CONFIG_DIR or ~/.kaggle. From an API token the engine mints the short-lived proxy token and renews it before it expires. Meant for prototypes and tests, not production traffic.

Parameters

• model (str): The model's Kaggle name, such as "google/gemini-3.6-flash".
• api_token (Optional[str], default None): A Kaggle API token.
• proxy_url (Optional[str], default None): The proxy's URL, with proxy_token.
• proxy_token (Optional[str], default None): A proxy token minted elsewhere, with proxy_url.
• api_base (Optional[str], default None): Another Kaggle API base URL for minting tokens (default https://api.kaggle.com); for tests.

Kaggle.spec#

def spec() -> Dict[str, Any]

The JSON spec the engine builds the model from (internal).

Vertex#

class Vertex:
    def __init__(
        self,
        model: str,
        project: Optional[str] = None,
        location: Optional[str] = None,
        token: Optional[str] = None,
        credentials_file: Optional[str] = None,
        api: str = 'auto',
        host: Optional[str] = None,
        token_url: Optional[str] = None,
        metadata_url: Optional[str] = None,
    )

Google Vertex AI, e.g. Vertex("google/gemini-3.6-flash", project="my-project", location="global") or "vertex:claude-sonnet-5".

Credentials, in order: token, credentials_file, else GOOGLE_APPLICATION_CREDENTIALS, the file that gcloud auth application-default login writes, or the metadata server when running on Google Cloud. Service-account key files are not supported. claude-... models go to Claude's endpoint, the others to the OpenAI-compatible one.

Parameters

• model (str): The model, such as "google/gemini-3.6-flash" or "claude-sonnet-5".
• project (Optional[str], default None): The Google Cloud project. Default: GOOGLE_CLOUD_PROJECT.
• location (Optional[str], default None): The region, or "global". Default: GOOGLE_CLOUD_LOCATION. There is no built-in default, because it decides where data is processed.
• token (Optional[str], default None): An OAuth access token obtained elsewhere.
• credentials_file (Optional[str], default None): A path to an application-default credentials file.
• api (str, default 'auto'): "auto" (by model name), "chat" (OpenAI-compatible) or "claude". Other values raise ValueError.
• host (Optional[str], default None): Another API host (a private endpoint).
• token_url (Optional[str], default None): Another OAuth token endpoint; for tests.
• metadata_url (Optional[str], default None): Another metadata-server URL; for tests.

Example

from criogenai import Agent, Vertex

model = Vertex("google/gemini-3.6-flash", project="my-project", location="europe-west4")
agent = Agent(model)

Vertex.spec#

def spec() -> Dict[str, Any]

The JSON spec the engine builds the model from (internal).

Bedrock#

class Bedrock:
    def __init__(
        self,
        model: str,
        region: Optional[str] = None,
        api_key: Optional[str] = None,
        api: str = 'auto',
        base_url: Optional[str] = None,
    )

Amazon Bedrock with an API key, e.g. Bedrock("us.anthropic.claude-sonnet-5", region="us-east-1") or "bedrock:us.anthropic.claude-sonnet-5".

Claude models are called over the Anthropic Messages API, other makers' models over Chat Completions. IAM roles (SigV4 signing) are not supported.

Parameters

• model (str): The Bedrock model id or inference profile.
• region (Optional[str], default None): The AWS region. Default: AWS_REGION or AWS_DEFAULT_REGION. There is no built-in default, so data never goes to a region you did not choose.
• api_key (Optional[str], default None): A Bedrock API key. Default: AWS_BEARER_TOKEN_BEDROCK. Without one, building an agent fails with ModelError.
• api (str, default 'auto'): "auto" (by model), "messages", "chat" or "responses". Other values raise ValueError.
• base_url (Optional[str], default None): Another endpoint, such as bedrock-mantle or a VPC endpoint.

Example

from criogenai import Agent, Bedrock

agent = Agent(Bedrock("us.anthropic.claude-sonnet-5", region="us-east-1"))

Bedrock.spec#

def spec() -> Dict[str, Any]

The JSON spec the engine builds the model from (internal).

Azure#

class Azure:
    def __init__(
        self,
        model: str,
        endpoint: Optional[str] = None,
        api_key: Optional[str] = None,
        token: Optional[str] = None,
        tenant_id: Optional[str] = None,
        client_id: Optional[str] = None,
        client_secret: Optional[str] = None,
        managed_identity: bool = False,
        api: str = 'responses',
        scope: Optional[str] = None,
        authority: Optional[str] = None,
        identity_endpoint: Optional[str] = None,
    )

Azure OpenAI or Microsoft Foundry: a deployment, e.g. Azure("gpt-5-mini", endpoint="my-resource") or "azure:gpt-5-mini".

Credentials, in order: api_key; token; client_secret with tenant_id and client_id (a service principal); managed_identity=True; else the environment: AZURE_OPENAI_API_KEY, or AZURE_TENANT_ID + AZURE_CLIENT_ID + AZURE_CLIENT_SECRET, or AZURE_USE_MANAGED_IDENTITY=1. The engine obtains Entra ID tokens and renews them.

Parameters

• model (str): The deployment name.
• endpoint (Optional[str], default None): The resource name or its URL. Default: AZURE_OPENAI_ENDPOINT.
• api_key (Optional[str], default None): The resource's API key.
• token (Optional[str], default None): A bearer token obtained elsewhere.
• tenant_id (Optional[str], default None): The Entra ID tenant, for a service principal. Default: AZURE_TENANT_ID.
• client_id (Optional[str], default None): The service principal's client id, or the client id of a user-assigned managed identity. Default for a service principal: AZURE_CLIENT_ID.
• client_secret (Optional[str], default None): The service principal's secret.
• managed_identity (bool, default False): Use the managed identity of the machine or app running this code.
• api (str, default 'responses'): "responses" (default) or "chat" (Chat Completions, for Foundry models from other makers). Other values raise ValueError.
• scope (Optional[str], default None): Another token scope (default https://cognitiveservices.azure.com/.default).
• authority (Optional[str], default None): Another Entra ID authority, such as a sovereign cloud's.
• identity_endpoint (Optional[str], default None): Another managed-identity endpoint; for tests.

Example

from criogenai import Agent, Azure

agent = Agent(Azure("gpt-5-mini", endpoint="my-resource", managed_identity=True))

Azure.spec#

def spec() -> Dict[str, Any]

The JSON spec the engine builds the model from (internal).

Ollama#

class Ollama(_Model)

Ollama on this machine (http://localhost:11434/v1), no key, e.g. Ollama("llama3.2") or "ollama:llama3.2".

Constructor: Ollama(model, api_key=None, base_url=None); pass base_url for Ollama on another address.

LMStudio#

class LMStudio(_Model)

LM Studio on this machine (http://localhost:1234/v1), no key.

Constructor: LMStudio(model, api_key=None, base_url=None); pass base_url for LM Studio on another address.

AnthropicCompatible#

class AnthropicCompatible(OpenAICompatible)

Any server that speaks Anthropic's Messages API: a self-hosted gateway, or a provider's Anthropic-style endpoint (DeepSeek, Z.ai, Qwen and Moonshot offer one).

Takes the same arguments as OpenAICompatible.

FakeReply#

class FakeReply:
    def __init__(self, spec: Dict[str, Any])

One scripted answer of a FakeModel: text, or tool calls.

Build replies with FakeReply.text, FakeReply.tool_call and FakeReply.tool_calls. A plain string in FakeModel's list is a text reply.

Parameters

• spec (Dict[str, Any]): The reply as the engine reads it (internal); use the static methods instead.

FakeReply.text#

@staticmethod
def text(text: str) -> 'FakeReply'

A text answer.

Parameters

• text (str): The answer's text.

FakeReply.tool_call#

@staticmethod
def tool_call(name: str, args: Optional[Dict[str, Any]] = None) -> 'FakeReply'

An answer that calls one tool.

Parameters

• name (str): The tool to call. For an agent with output=, the name "final_output" gives the structured output.
• args (Optional[Dict[str, Any]], default None): The call's arguments. None means {}.

FakeReply.tool_calls#

@staticmethod
def tool_calls(calls: Sequence[Tuple[str, Dict[str, Any]]]) -> 'FakeReply'

An answer that calls several tools at once.

Parameters

• calls (Sequence[Tuple[str, Dict[str, Any]]]): (name, args) pairs, in call order.

Embedder#

class Embedder:
    def __init__(self, spec: Dict[str, Any])

Turns text into vectors, for search by meaning.

Used by Memory.semantic, collections and Gateway.embed. Build one with the static methods; keys come from the same environment variables as the models'. repr() never shows a key.

Parameters

• spec (Dict[str, Any]): The embedder as the engine reads it (internal); use the static methods instead.

Example

from criogenai import Embedder, Gateway

vectors = Gateway().embed_sync(Embedder.fake(), ["a red car", "a blue car"])
print(len(vectors), len(vectors[0]))  # 2 256

Embedder.openai#

@staticmethod
def openai(
    model: str = 'text-embedding-3-small',
    api_key: Optional[str] = None,
    dimensions: Optional[int] = None,
) -> 'Embedder'

OpenAI's /embeddings.

Parameters

• model (str, default 'text-embedding-3-small'): The embedding model.
• api_key (Optional[str], default None): The key. Default: OPENAI_API_KEY.
• dimensions (Optional[int], default None): Shorter vectors, for models that support it. None keeps the model's size.

Raises

• ValidationError: No key given or found (invalid_value; also a ValueError).

Embedder.gemini#

@staticmethod
def gemini(
    model: str = 'gemini-embedding-001',
    api_key: Optional[str] = None,
) -> 'Embedder'

Gemini's OpenAI-compatible /embeddings.

Parameters

• model (str, default 'gemini-embedding-001'): The embedding model.
• api_key (Optional[str], default None): The key. Default: GEMINI_API_KEY.

Raises

• ValidationError: No key given or found (invalid_value; also a ValueError).

Embedder.ollama#

@staticmethod
def ollama(
    model: str = 'nomic-embed-text',
    base_url: Optional[str] = None,
) -> 'Embedder'

Ollama's /embeddings on this machine: nothing leaves it.

Parameters

• model (str, default 'nomic-embed-text'): The embedding model, pulled into Ollama first.
• base_url (Optional[str], default None): Ollama on another address. Default http://localhost:11434/v1.

Embedder.openai_compatible#

@staticmethod
def openai_compatible(
    model: str,
    base_url: str,
    api_key: Optional[str] = None,
    dimensions: Optional[int] = None,
) -> 'Embedder'

Any OpenAI-compatible /embeddings (vLLM, a gateway).

Parameters

• model (str): The embedding model name the server expects.
• base_url (str): The server's API URL, usually ending in /v1.
• api_key (Optional[str], default None): The key, if the server needs one.
• dimensions (Optional[int], default None): Shorter vectors, for models that support it.

Embedder.fake#

@staticmethod
def fake(synonyms: Optional[Dict[str, str]] = None, failing: bool = False) -> 'Embedder'

An embedder for tests; it never makes a network call.

Texts that share words get close vectors.

Parameters

• synonyms (Optional[Dict[str, str]], default None): Words to count as the same word, such as {"car": "automobile"}.
• failing (bool, default False): Make every call fail, to test the fallback: memory search then falls back to keywords.