Letta Model Configuration

Patterns for configuring LLM models on Letta agents via SDK/API. Covers model handles, settings, provider-specific configuration, and custom endpoints.

When to Use This Skill

Use this skill when:

• Creating agents with specific model configurations

• Adjusting model settings (temperature, max tokens, context window)

• Configuring provider-specific features (OpenAI reasoning, Anthropic thinking)

• Setting up custom OpenAI-compatible endpoints

• Changing models on existing agents

• Configuring embedding models for self-hosted deployments

Not covered here: Model selection advice (which model to choose) - see agent-development skill's references/model-recommendations.md .

Model Handles

Models use a provider/model-name format:

Provider Handle Prefix Example

OpenAI openai/

openai/gpt-4o , openai/gpt-4o-mini

Anthropic anthropic/

anthropic/claude-sonnet-4-5-20250929

Google AI google_ai/

google_ai/gemini-2.0-flash

Azure OpenAI azure/

azure/gpt-4o

AWS Bedrock bedrock/

bedrock/anthropic.claude-3-5-sonnet

Groq groq/

groq/llama-3.3-70b-versatile

Together together/

together/meta-llama/Llama-3-70b

OpenRouter openrouter/

openrouter/anthropic/claude-3.5-sonnet

Ollama (local) ollama/

ollama/llama3.2

Basic Model Configuration

Python

from letta_client import Letta

client = Letta(api_key="your-api-key")

agent = client.agents.create( model="openai/gpt-4o", model_settings={ "provider_type": "openai", # Required - must match model provider "temperature": 0.7, "max_output_tokens": 4096, }, context_window_limit=128000 )

TypeScript

import Letta from "@letta-ai/letta-client";

const client = new Letta({ apiKey: "your-api-key" });

const agent = await client.agents.create({ model: "openai/gpt-4o", model_settings: { provider_type: "openai", // Required - must match model provider temperature: 0.7, max_output_tokens: 4096, }, context_window_limit: 128000, });

Common Settings

Setting Type Description

provider_type

string Required. Must match model provider (openai , anthropic , google_ai , etc.)

temperature

float Controls randomness (0.0-2.0). Lower = more deterministic.

max_output_tokens

int Maximum tokens in the response.

Context Window Limit

Set at agent level (not inside model_settings ):

agent = client.agents.create( model="anthropic/claude-sonnet-4-5-20250929", context_window_limit=200000 # Use 200K of Claude's context )

Important:

• Must be <= model's maximum context size

• Default: 32,000 tokens if not specified

• Larger windows increase latency and may reduce reliability

• When context fills up, Letta automatically summarizes older messages

Changing an Agent's Model

Update existing agents with agents.update() :

Python

Change model only

client.agents.update( agent_id=agent.id, model="anthropic/claude-sonnet-4-5-20250929" )

Change model and settings

client.agents.update( agent_id=agent.id, model="openai/gpt-4o", model_settings={ "provider_type": "openai", "temperature": 0.5 }, context_window_limit=64000 )

TypeScript

// Change model only await client.agents.update(agent.id, { model: "anthropic/claude-sonnet-4-5-20250929", });

// Change model and settings await client.agents.update(agent.id, { model: "openai/gpt-4o", model_settings: { provider_type: "openai", temperature: 0.5, }, context_window_limit: 64000, });

Note: Agents retain memory and tools when changing models.

Provider-Specific Settings

For OpenAI reasoning models and Anthropic extended thinking, see references/provider-settings.md .

Custom Endpoints

For OpenAI-compatible endpoints (vLLM, LM Studio, LocalAI), see references/custom-endpoints.md .

Embedding Models

Required for self-hosted deployments (Letta Cloud handles automatically):

agent = client.agents.create( model="openai/gpt-4o", embedding="openai/text-embedding-3-small" )

Common embedding models:

• openai/text-embedding-3-small (recommended)

• openai/text-embedding-3-large

• openai/text-embedding-ada-002

Anti-Hallucination Checklist

Before configuring models, verify:

• Model handle uses correct provider/model-name format

• model_settings includes required provider_type field

• context_window_limit is set at agent level, not in model_settings

• Provider-specific settings use correct nested structure (see references)

• For self-hosted: embedding model is specified

• Temperature is within valid range (0.0-2.0)

Example Scripts

See scripts/ for runnable examples:

• scripts/basic_config.py

• Basic model configuration

• scripts/basic_config.ts

• TypeScript equivalent

• scripts/change_model.py

• Changing models on existing agents

• scripts/provider_specific.py

• OpenAI reasoning, Anthropic thinking