Arize Trace Skill

Concepts

• Trace = a tree of spans sharing a context.trace_id, rooted at a span with parent_id = null
• Span = a single operation (LLM call, tool call, retriever, chain, agent)
• Session = a group of traces sharing attributes.session.id (e.g., a multi-turn conversation)

Use ax spans export to download trace data. This is the only supported command for retrieving spans.

Resolving project for export: The PROJECT positional argument accepts either a project name or a base64 project ID. When using a name, --space-id is required. If you hit limit errors or 401 Unauthorized when using a project name, resolve it to a base64 ID: run ax projects list --space-id SPACE_ID -l 100 -o json, find the project by name, and use its id as PROJECT.

Exploratory export rule: When exporting spans or traces without a specific --trace-id, --span-id, or --session-id (i.e., browsing/exploring a project), always start with -l 50 to pull a small sample first. Summarize what you find, then pull more data only if the user asks or the task requires it. This avoids slow queries and overwhelming output on large projects.

Default output directory: Always use --output-dir .arize-tmp-traces on every ax spans export call. The CLI automatically creates the directory and adds it to .gitignore.

Prerequisites

Three things are needed: ax CLI, an API key (env var or profile), and a space ID. A project name or ID is also needed but usually comes from the user's message.

Install ax

Verify ax is installed and working before proceeding:

1. Check if ax is on PATH: command -v ax (Unix) or where ax (Windows)
2. If not found, check common install locations:
   • macOS/Linux: test -x ~/.local/bin/ax && export PATH="$HOME/.local/bin:$PATH"
   • Windows: check %APPDATA%\Python\Scripts\ax.exe or %LOCALAPPDATA%\Programs\Python\Scripts\ax.exe
3. If still not found, install it (requires shell access to install packages):
   • Preferred: uv tool install arize-ax-cli
   • Alternative: pipx install arize-ax-cli
   • Fallback: pip install arize-ax-cli
4. After install, if ax is not on PATH:
   • macOS/Linux: export PATH="$HOME/.local/bin:$PATH"
   • Windows (PowerShell): $env:PATH = "$env:APPDATA\Python\Scripts;$env:PATH"
5. If ax --version fails with an SSL/certificate error:
   • macOS: export SSL_CERT_FILE=/etc/ssl/cert.pem
   • Linux: export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
   • Windows (PowerShell): $env:SSL_CERT_FILE = "C:\Program Files\Common Files\SSL\cert.pem" (or use python -c "import certifi; print(certifi.where())" to find the cert bundle)
6. ax --version must succeed before proceeding. If it doesn't, stop and ask the user for help.

Verify environment

Run a quick check for credentials:

macOS/Linux (bash):

ax --version && echo "--- env ---" && if [ -n "$ARIZE_API_KEY" ]; then echo "ARIZE_API_KEY: (set)"; else echo "ARIZE_API_KEY: (not set)"; fi && echo "ARIZE_SPACE_ID: ${ARIZE_SPACE_ID:-(not set)}" && echo "--- profiles ---" && ax profiles show 2>&1

Windows (PowerShell):

ax --version; Write-Host "--- env ---"; Write-Host "ARIZE_API_KEY: $(if ($env:ARIZE_API_KEY) { '(set)' } else { '(not set)' })"; Write-Host "ARIZE_SPACE_ID: $env:ARIZE_SPACE_ID"; Write-Host "--- profiles ---"; ax profiles show 2>&1

Read the output and proceed immediately if either the env var or the profile has an API key. Only ask the user if both are missing. Resolve failures:

• No API key in env and no profile → AskQuestion: "Arize API key (https://app.arize.com/admin > API Keys)"
• Space ID unknown → AskQuestion, or run ax projects list -o json --limit 100 --space-id $ARIZE_SPACE_ID and present as selectable options
• Project unclear → ask, or run ax projects list -o json --limit 100 and search for a match

IMPORTANT: --space-id is required when using a human-readable project name as the PROJECT positional argument. It is not needed when using a base64-encoded project ID. If you hit 401 Unauthorized or limit errors when using a project name, resolve it to a base64 ID first (see "Resolving project for export" in Concepts).

Export Spans: ax spans export

The primary command for downloading trace data to a file.

By trace ID

ax spans export PROJECT_ID --trace-id TRACE_ID --output-dir .arize-tmp-traces

By span ID

ax spans export PROJECT_ID --span-id SPAN_ID --output-dir .arize-tmp-traces

By session ID

ax spans export PROJECT_ID --session-id SESSION_ID --output-dir .arize-tmp-traces

Flags

Exactly one of --trace-id, --span-id, --session-id is required.

Output is a JSON array of span objects. File naming: {type}_{id}_{timestamp}/spans.json.

Bulk export with --all (Arrow Flight)

By default, ax spans export uses the REST API which is limited to 500 spans per page and capped by --limit. Pass --all to switch to Arrow Flight for streaming bulk export with no span limit.

ax spans export PROJECT_ID --space-id SPACE_ID --filter "status_code = 'ERROR'" --all --output-dir .arize-tmp-traces

REST vs Flight trade-offs:

• REST (default): Lower friction -- no Arrow/Flight dependency needed, uses standard HTTPS ports, works through any corporate proxy or firewall. Limited to 500 spans per page.
• Flight (--all): Required for bulk export beyond 500 spans. Uses gRPC+TLS on a separate host/port which some corporate networks may block.

When to use --all:

• Exporting more than 500 spans
• Downloading full traces with many child spans
• Large time-range exports

Agent auto-escalation rule: If a REST export returns exactly the number of spans requested by -l (or 500 if no limit was set), the result is likely truncated. Increase -l or re-run with --all to get the full dataset — but only when the user asks or the task requires more data.

Requirements for --all:

• --space-id is required (Flight uses space_id + project_name, not project_id)
• --limit is ignored when --all is set

Networking notes for --all: Arrow Flight connects to flight.arize.com:443 via gRPC+TLS -- this is a different host from the REST API (api.arize.com). On internal or private networks, the Flight endpoint may use a different host/port. Configure via:

• ax profile: flight_host, flight_port, flight_scheme
• Environment variables: ARIZE_FLIGHT_HOST, ARIZE_FLIGHT_PORT, ARIZE_FLIGHT_SCHEME

The --all flag is also available on ax traces export, ax datasets export, and ax experiments export with the same behavior (REST by default, Flight with --all).

Export Traces: ax traces export

Export full traces -- all spans belonging to traces that match a filter. Uses a two-phase approach:

1. Phase 1: Find spans matching --filter (up to --limit via REST, or all via Flight with --all)
2. Phase 2: Extract unique trace IDs, then fetch every span for those traces

# Explore recent traces (start small with -l 50, pull more if needed)
ax traces export PROJECT_ID -l 50 --output-dir .arize-tmp-traces

# Export traces with error spans (REST, up to 500 spans in phase 1)
ax traces export PROJECT_ID --filter "status_code = 'ERROR'" --stdout

# Export all traces matching a filter via Flight (no limit)
ax traces export PROJECT_ID --space-id SPACE_ID --filter "status_code = 'ERROR'" --all

Flags

How it differs from ax spans export

• ax spans export exports individual spans matching a filter
• ax traces export exports complete traces -- it finds spans matching the filter, then pulls ALL spans for those traces (including siblings and children that may not match the filter)

Filter Syntax Reference

SQL-like expressions passed to --filter.

Common filterable columns

Operators

=, !=, <, <=, >, >=, AND, OR, IN, CONTAINS, LIKE, IS NULL, IS NOT NULL

Examples

status_code = 'ERROR'
latency_ms > 5000
name = 'ChatCompletion' AND status_code = 'ERROR'
attributes.llm.model_name = 'gpt-4o'
attributes.openinference.span.kind IN ('LLM', 'AGENT')
attributes.error.type LIKE '%Transport%'
event.attributes CONTAINS 'TimeoutError'

Tips

• Prefer IN over multiple OR conditions: name IN ('a', 'b', 'c') not name = 'a' OR name = 'b' OR name = 'c'
• Start broad with LIKE, then switch to = or IN once you know exact values
• Use CONTAINS for event.attributes (error tracebacks) -- exact match is unreliable on complex text
• Always wrap string values in single quotes

Workflows

Debug a failing trace

1. ax traces export PROJECT_ID --filter "status_code = 'ERROR'" -l 50 --output-dir .arize-tmp-traces
2. Read the output file, look for spans with status_code: ERROR
3. Check attributes.error.type and attributes.error.message on error spans

Download a conversation session

1. ax spans export PROJECT_ID --session-id SESSION_ID --output-dir .arize-tmp-traces
2. Spans are ordered by start_time, grouped by context.trace_id
3. If you only have a trace_id, export that trace first, then look for attributes.session.id in the output to get the session ID

Export for offline analysis

ax spans export PROJECT_ID --trace-id TRACE_ID --output-dir .arize-tmp-traces --stdout | jq '.[]'

Span Column Reference (OpenInference Semantic Conventions)

Core Identity and Timing

Where to Find Prompts and LLM I/O

Generic input/output (all span kinds):

LLM-specific message arrays (structured chat format):

Prompt templates:

Finding prompts by span kind:

• LLM span: Check attributes.llm.input_messages for structured chat messages, OR attributes.input.value for serialized prompt. Check attributes.llm.prompt_template.template for the template.
• Chain/Agent span: Check attributes.input.value for the user's question. Actual LLM prompts are on child LLM spans.
• Tool span: Check attributes.input.value for tool input, attributes.output.value for tool result.

LLM Model and Cost

Tool Spans

Retriever Spans

Reranker Spans

Session, User, and Custom Metadata

Errors and Exceptions

Evaluations and Annotations

Embeddings

Troubleshooting

Save Credentials

At session end, if the user manually provided an API key, space ID, or project ID (not loaded from an existing profile), offer to save them to ~/.arize/config.toml. Use AskQuestion with "Yes, save them" / "No thanks". Skip if all values were already in the profile.

Read the existing file (or create it), add/update only the new fields, and write it back:

[auth]
api_key = "THE_API_KEY"

[defaults]
space_id = "THE_SPACE_ID"
project = "THE_PROJECT_NAME"