ModelBackendSettings

Model connection configuration.

Quick Example

from mamba_agents.config import ModelBackendSettings

# Via AgentSettings
from mamba_agents import AgentSettings
settings = AgentSettings(
    model_backend={
        "base_url": "http://localhost:11434/v1",
        "model": "llama3.2",
        "temperature": 0.7,
    }
)

Configuration Options

Option	Type	Default	Description
model	str	"llama3.2"	Model identifier
base_url	str	"http://localhost:11434/v1"	API endpoint
api_key	SecretStr	None	API key
temperature	float	0.7	Sampling temperature
max_tokens	int	None	Max output tokens
timeout	float	30.0	Request timeout
max_retries	int	3	Retry attempts

Environment Variables

MAMBA_MODEL_BACKEND__BASE_URL=https://api.openai.com/v1
MAMBA_MODEL_BACKEND__MODEL=gpt-4o
MAMBA_MODEL_BACKEND__API_KEY=sk-...
MAMBA_MODEL_BACKEND__TEMPERATURE=0.7

API Reference

ModelBackendSettings

Bases: BaseModel

Configuration for the model backend connection.

This configuration defines how to connect to an OpenAI-compatible API endpoint, which can be a local LLM server (Ollama, vLLM, llama.cpp) or a remote service.

ATTRIBUTE	DESCRIPTION
base_url	Base URL for the OpenAI-compatible API endpoint. TYPE: str
api_key	Optional API key for authenticated endpoints. TYPE: SecretStr | None
model	Model identifier to use for requests. TYPE: str
timeout	Request timeout in seconds. TYPE: float
max_retries	Maximum retry attempts for failed requests. TYPE: int
temperature	Sampling temperature for generation. TYPE: float
max_tokens	Maximum tokens to generate (None for model default). TYPE: int | None

get_headers

get_headers() -> dict[str, str]

Get HTTP headers for API requests.

RETURNS	DESCRIPTION
dict[str, str]	Dictionary of headers including Authorization if API key is set.

Source code in src/mamba_agents/config/model_backend.py

def get_headers(self) -> dict[str, str]:
    """Get HTTP headers for API requests.

    Returns:
        Dictionary of headers including Authorization if API key is set.
    """
    headers: dict[str, str] = {}
    if self.api_key:
        headers["Authorization"] = f"Bearer {self.api_key.get_secret_value()}"
    return headers