Hugging Face

Configuration for the Hugging Face model provider.

A provider block with kind = "huggingface" configures the Hugging Face provider and its router for chat completion models. Model names use the identifiers accepted by Hugging Face, such as google/gemma-2-2b-it or meta-llama/Meta-Llama-3.1-8B-Instruct.

Fields

Field	Type	Required	runtime()	Default	Description
`kind`	`huggingface`	yes	no	…	Provider kind.
`models`	array<string \| ModelEntry>	no	no	unset	Models permitted for this provider, with optional model-specific inference settings.
`config`	ProviderConfig	no		unset	API credentials and endpoint settings for this provider.
`params`	InferenceParams	no		unset	Default inference settings for requests to this provider.

When config.api_key is unset, the key is read from the HUGGINGFACE_API_KEY environment variable. The agent fails to start when neither is set. When config.base_url is unset, the default Hugging Face router endpoint is used.

ModelEntry

Field	Type	Required	runtime()	Default	Description
`name`	string	yes	no	…	Model identifier.
`params`	InferenceParams	no		unset	Model-specific inference parameter overrides.

ProviderConfig

Field	Type	Required	runtime()	Default	Description
`api_key`	string	no	yes	unset	Hugging Face API token. Wrap in `secret(runtime(...))` to keep it out of build artifacts.
`base_url`	string	no	yes	unset	Hugging Face router or compatible API base URL.
`headers`	map<string, HeaderValue>	no	yes	`{}`	HTTP headers sent with every model request, keyed by header name. Header names are case insensitive.

HeaderValue

A header value is one of:

Form	Header value
a string	The string, unchanged.
`expression("source")`	The result of the CEL expression `source`, evaluated for each model request.

config {
  headers = {
    "X-Team"      = "platform"
    "X-Tenant-Id" = expression("ctx.tenant_id")
  }
}

For the values an expression can read and how evaluation fails, see Send custom headers.

InferenceParams

The params block sets inference parameter defaults for all models under this provider. All fields are optional and support runtime().

Field	Type	Required	runtime()	Default	Description
`max_tokens`	integer	no	yes	unset	Maximum number of tokens to generate.
`temperature`	number	no	yes	unset	Controls randomness when selecting the next token; higher values produce more varied output.
`top_p`	number	no	yes	unset	Selects tokens from the smallest group whose combined probability reaches this value.
`top_k`	integer	no	yes	unset	Limits the next-token choices to the K most likely tokens.
`stop_sequences`	array<string>	no	yes	unset	Sequences that stop generation.
`frequency_penalty`	number	no	yes	unset	Reduces repetition by penalizing tokens according to how often they have appeared.
`presence_penalty`	number	no	yes	unset	Penalizes tokens that have already appeared to reduce repetition.
`seed`	integer	no	yes	unset	Seed for deterministic sampling.
`provider_params`	object	no	yes	unset	Provider-specific parameters passed directly to the API.

Example

agent.acl

provider "huggingface" {
  kind = "huggingface"
  models = [
    "google/gemma-2-2b-it",
    {
      name = "meta-llama/Meta-Llama-3.1-8B-Instruct"
      params {
        max_tokens  = 8192
        temperature = 0.7
      }
    }
  ]

  config {
    api_key  = secret(runtime("HUGGINGFACE_API_KEY"))
    base_url = runtime("HUGGINGFACE_BASE_URL", "https://router.huggingface.co")
  }

  params {
    max_tokens  = 4096
    temperature = 0.5
  }
}

agent "assistant" {
  graph {
    type = "react"
  }

  model {
    provider = "huggingface"
    name     = "google/gemma-2-2b-it"
  }
}

tip

Always wrap API tokens in secret(runtime(...)). This prevents the token from appearing in build artifacts and the output of agentc inspect.

Fields

Field

Type

Required

runtime()

Default

Description

kind

huggingface

yes

…

Provider kind.

models

array<string | ModelEntry>

unset

Models permitted for this provider, with optional model-specific inference settings.

config

ProviderConfig

unset

API credentials and endpoint settings for this provider.

params

InferenceParams

unset

Default inference settings for requests to this provider.

Field

Type

Required

runtime()

Default

Description

name

string

yes

…

Model identifier.

params

InferenceParams

unset

Model-specific inference parameter overrides.

ProviderConfig

Field

Type

Required

runtime()

Default

Description

api_key

string

yes

unset

Hugging Face API token. Wrap in secret(runtime(...)) to keep it out of build artifacts.

base_url

string

yes

unset

Hugging Face router or compatible API base URL.

headers

map<string, HeaderValue>

yes

{}

HTTP headers sent with every model request, keyed by header name. Header names are case insensitive.

HeaderValue

A header value is one of:

Form

Header value

a string

The string, unchanged.

expression("source")

The result of the CEL expression source, evaluated for each model request.

config {
  headers = {
    "X-Team"      = "platform"
    "X-Tenant-Id" = expression("ctx.tenant_id")
  }
}

For the values an expression can read and how evaluation fails, see Send custom headers.

InferenceParams

The params block sets inference parameter defaults for all models under this provider. All fields are optional and support runtime().

Field

Type

Required

runtime()

Default

Description

max_tokens

integer

yes

unset

Maximum number of tokens to generate.

temperature

number

yes

unset

Controls randomness when selecting the next token; higher values produce more varied output.

top_p

number

yes

unset

Selects tokens from the smallest group whose combined probability reaches this value.

top_k

integer

yes

unset

Limits the next-token choices to the K most likely tokens.

stop_sequences

array<string>

yes

unset

Sequences that stop generation.

frequency_penalty

number

yes

unset

Reduces repetition by penalizing tokens according to how often they have appeared.

presence_penalty

number

yes

unset

Penalizes tokens that have already appeared to reduce repetition.

seed

integer

yes

unset

Seed for deterministic sampling.

provider_params

object

yes

unset

Provider-specific parameters passed directly to the API.

Example

agent.acl

provider "huggingface" {
  kind = "huggingface"
  models = [
    "google/gemma-2-2b-it",
    {
      name = "meta-llama/Meta-Llama-3.1-8B-Instruct"
      params {
        max_tokens  = 8192
        temperature = 0.7
      }
    }
  ]

  config {
    api_key  = secret(runtime("HUGGINGFACE_API_KEY"))
    base_url = runtime("HUGGINGFACE_BASE_URL", "https://router.huggingface.co")
  }

  params {
    max_tokens  = 4096
    temperature = 0.5
  }
}

agent "assistant" {
  graph {
    type = "react"
  }

  model {
    provider = "huggingface"
    name     = "google/gemma-2-2b-it"
  }
}

tip

Always wrap API tokens in secret(runtime(...)). This prevents the token from appearing in build artifacts and the output of agentc inspect.