Ollama

The ollama sub-block configures a local Ollama runtime. Ollama runs entirely on the local machine and does not require an API key.

Fields

The config block is optional. When omitted, the client connects to http://localhost:11434, which is the default address of a locally running Ollama instance.

params

The params block sets inference parameter defaults for all models under this provider. All fields are optional and support runtime().

Example

providers {
  ollama {
    models = [
      "llama3.2",
      {
        name = "mistral"
        params {
          temperature = 0.9
          seed        = 42
        }
      }
    ]

    config {
      base_url = runtime("OLLAMA_BASE_URL", "http://localhost:11434")
    }

    params {
      max_tokens  = 2048
      temperature = 0.6
    }
  }
}

In this example, requests to llama3.2 use the provider-level defaults. Requests to mistral use temperature = 0.9 and seed = 42 from its model-level params, while inheriting max_tokens from the provider-level defaults.