Configure a model provider

An agent talks to a model through a provider. Each provider block declares one provider: its kind, how to connect to it, and which models it serves. The agent's model block picks one by its label. Inference parameters such as temperature and token limits live with the provider, not the agent. This guide covers the common setup. For every kind and its fields, see the provider reference.

Declare a provider and select its model

Declare the provider, read its API key from the environment, and point the agent's model block at it. provider is the label of the provider block, and name must be one of its models:

provider "anthropic" {
  kind   = "anthropic"
  models = ["claude-haiku-4-5", "claude-sonnet-4-6"]

  config {
    api_key = secret(runtime("ANTHROPIC_API_KEY"))
  }
}

agent "assistant" {
  graph {
    type = "react"
  }

  model {
    provider = "anthropic"
    name     = "claude-haiku-4-5"
  }
}

The label is yours to choose. Here it matches the kind, which is the convention when a manifest has one provider of each kind.

Use two endpoints of the same kind

Declare one provider block per endpoint, each with its own label. Both blocks below use the openai kind: one connects to OpenAI, the other to a self-hosted server with an OpenAI-compatible API. Each has its own key, models, and parameters:

provider "cloud" {
  kind   = "openai"
  models = ["gpt-4o", "gpt-4o-mini"]

  config {
    api_key = secret(runtime("OPENAI_API_KEY"))
  }
}

provider "local" {
  kind   = "openai"
  models = ["qwen3-32b"]

  config {
    base_url = runtime("LOCAL_LLM_URL", "http://127.0.0.1:8000/v1")
    api_key  = secret(runtime("LOCAL_LLM_KEY", "unused"))
  }

  params {
    temperature = 0.2
  }
}

agent "assistant" {
  graph {
    type = "react"
  }

  model {
    provider = "local"
    name     = "qwen3-32b"
  }
}

Errors from either endpoint name its label, so cloud and local stay distinguishable in logs.

Choose the model at deployment time

The provider and name fields accept runtime(), so a single binary can run different models per environment without recompiling:

model {
  provider = runtime("AGENT_PROVIDER", "cloud")
  name     = runtime("AGENT_MODEL", "gpt-4o-mini")
}

Set inference parameters

Add a params block to a provider to set defaults for every request made through it:

provider "anthropic" {
  kind   = "anthropic"
  models = ["claude-haiku-4-5"]

  config {
    api_key = secret(runtime("ANTHROPIC_API_KEY"))
  }

  params {
    max_tokens  = 4096
    temperature = 0.5
  }
}

All parameter fields are optional and fall back to the provider's built-in defaults. The full set, including top_p, top_k, stop_sequences, and others, is on each kind's page in the provider reference.

Override parameters for one model

To vary parameters by model, promote a models entry from a plain string to an object with its own params. Model-level values are merged over the provider defaults, so you override only what you name:

provider "anthropic" {
  kind = "anthropic"
  models = [
    "claude-haiku-4-5",
    {
      name = "claude-opus-4-7"
      params {
        max_tokens  = 16000
        temperature = 0.8
      }
    }
  ]

  config {
    api_key = secret(runtime("ANTHROPIC_API_KEY"))
  }

  params {
    max_tokens  = 4096
    temperature = 0.5
  }
}

Requests to claude-haiku-4-5 use the provider defaults; requests to claude-opus-4-7 use its own.

Pass provider-specific options

For options outside the standard set, such as Anthropic extended thinking, use provider_params, which is passed straight through to the provider API:

params {
  provider_params = {
    thinking = {
      type          = "enabled"
      budget_tokens = 5000
    }
  }
}

Send custom headers

To send extra HTTP headers with each request to the model API, such as a team tag for a gateway, set headers in the provider's config block:

provider "cloud" {
  kind   = "openai"
  models = ["gpt-4o"]

  config {
    api_key = secret(runtime("OPENAI_API_KEY"))

    headers = {
      "X-Team" = "platform"
    }
  }
}

Compute a header for each request

Wrap a CEL expression in expression() to compute a header value for each model request. The expression must evaluate to a string:

headers = {
  "X-Team"      = "platform"
  "X-Tenant-Id" = expression("ctx.tenant_id")
}

An expression reads the model request's context through the variable ctx:

The agent's graph decides which of these values a model request carries. A graph can attach further values under ctx.attributes, such as a run identifier at ctx.attributes.run_id:

headers = {
  "X-Request-Tag" = expression("'run-' + ctx.attributes.run_id")
}

Handle missing values

Use has() to read a value only when it is present:

"X-Run" = expression("has(ctx.attributes.run_id) ? ctx.attributes.run_id : 'none'")

Set headers at deployment time

headers accepts runtime(). The environment value is a JSON object that maps each header name to its value form:

headers = runtime("PROVIDER_HEADERS", {
  "X-Team" = "platform"
})

PROVIDER_HEADERS='{"X-Team":{"syntax":"literal","source":"platform"},"X-Tenant-Id":{"syntax":"cel","source":"ctx.tenant_id"}}'

Where to go next

• Agents and prompts: how model selection fits the rest of the agent.
• provider reference: the provider block and every kind's fields.
• agent reference: the model block.