Building Chat Widgets

Create interactive widgets for AI chat with actions and entity tagging.

Quick Start

const chatkit = useChatKit({ api: { url: API_URL, domainKey: DOMAIN_KEY },

widgets: { onAction: async (action, widgetItem) => { if (action.type === "view_details") { navigate(/details/${action.payload.id}); } }, }, });

Action Handler Types

Handler Defined In Processed By Use Case

"client"

Widget template Frontend onAction

Navigation, local state

"server"

Widget template Backend action()

Data mutation, widget replacement

Widget Lifecycle

1. Agent tool generates widget → yield WidgetItem
2. Widget renders in chat with action buttons
3. User clicks action → action dispatched
4. Handler processes action:
   • client: onAction callback in frontend
   • server: action() method in ChatKitServer
5. Optional: Widget replaced with updated state

Core Patterns

1. Widget Templates

Define reusable widget layouts with dynamic data:

{ "type": "ListView", "children": [ { "type": "ListViewItem", "key": "item-1", "onClickAction": { "type": "item.select", "handler": "client", "payload": { "itemId": "item-1" } }, "children": [ { "type": "Row", "gap": 3, "children": [ { "type": "Icon", "name": "check", "color": "success" }, { "type": "Text", "value": "Item title", "weight": "semibold" } ] } ] } ] }

2. Client-Handled Actions

Actions that update local state, navigate, or send follow-up messages:

Widget Definition:

{ "type": "Button", "label": "View Article", "onClickAction": { "type": "open_article", "handler": "client", "payload": { "id": "article-123" } } }

Frontend Handler:

const chatkit = useChatKit({ api: { url: API_URL, domainKey: DOMAIN_KEY },

widgets: { onAction: async (action, widgetItem) => { switch (action.type) { case "open_article": navigate(/article/${action.payload?.id}); break;

    case "more_suggestions":
      await chatkit.sendUserMessage({ text: "More suggestions, please" });
      break;

    case "select_option":
      setSelectedOption(action.payload?.optionId);
      break;
  }
},

}, });

3. Server-Handled Actions

Actions that mutate data, update widgets, or require backend processing:

Widget Definition:

{ "type": "ListViewItem", "onClickAction": { "type": "line.select", "handler": "server", "payload": { "id": "blue-line" } } }

Backend Handler:

from chatkit.types import ( Action, WidgetItem, ThreadItemReplacedEvent, ThreadItemDoneEvent, AssistantMessageItem, ClientEffectEvent, )

class MyServer(ChatKitServer[dict]):

async def action(
    self,
    thread: ThreadMetadata,
    action: Action[str, Any],
    sender: WidgetItem | None,
    context: RequestContext,  # Note: Already RequestContext, not dict
) -> AsyncIterator[ThreadStreamEvent]:

    if action.type == "line.select":
        line_id = action.payload["id"]  # Use .payload, not .arguments

        # 1. Update widget with selection
        updated_widget = build_selector_widget(selected=line_id)
        yield ThreadItemReplacedEvent(
            item=sender.model_copy(update={"widget": updated_widget})
        )

        # 2. Stream assistant message
        yield ThreadItemDoneEvent(
            item=AssistantMessageItem(
                id=self.store.generate_item_id("msg", thread, context),
                thread_id=thread.id,
                created_at=datetime.now(),
                content=[{"text": f"Selected {line_id}"}],
            )
        )

        # 3. Trigger client effect
        yield ClientEffectEvent(
            name="selection_changed",
            data={"lineId": line_id},
        )

4. Entity Tagging (@mentions)

Allow users to @mention entities in messages:

const chatkit = useChatKit({ api: { url: API_URL, domainKey: DOMAIN_KEY },

entities: { onTagSearch: async (query: string): Promise<Entity[]> => { const results = await fetch(/api/search?q=${query}).then(r => r.json());

  return results.map((item) => ({
    id: item.id,
    title: item.name,
    icon: item.type === "person" ? "profile" : "document",
    group: item.type === "People" ? "People" : "Articles",
    interactive: true,
    data: { type: item.type, article_id: item.id },
  }));
},

onClick: (entity: Entity) => {
  if (entity.data?.article_id) {
    navigate(`/article/${entity.data.article_id}`);
  }
},

}, });

5. Composer Tools (Mode Selection)

Let users select different AI modes from the composer:

const TOOL_CHOICES = [ { id: "general", label: "Chat", icon: "sparkle", placeholderOverride: "Ask anything...", pinned: true, }, { id: "event_finder", label: "Find Events", icon: "calendar", placeholderOverride: "What events are you looking for?", pinned: true, }, ];

const chatkit = useChatKit({ api: { url: API_URL, domainKey: DOMAIN_KEY }, composer: { placeholder: "What would you like to do?", tools: TOOL_CHOICES, }, });

Backend Routing:

async def respond(self, thread, item, context): tool_choice = context.metadata.get("tool_choice")

if tool_choice == "event_finder":
    agent = self.event_finder_agent
else:
    agent = self.general_agent

result = Runner.run_streamed(agent, input_items)
async for event in stream_agent_response(context, result):
    yield event

Widget Component Reference

Layout Components

Component Props Description

ListView

children

Scrollable list container

ListViewItem

key , onClickAction , children

Clickable list item

Row

gap , align , justify , children

Horizontal flex

Col

gap , padding , children

Vertical flex

Box

size , radius , background , padding

Styled container

Content Components

Component Props Description

Text

value , size , weight , color

Text display

Title

value , size , weight

Heading text

Image

src , alt , width , height

Image display

Icon

name , size , color

Icon from set

Interactive Components

Component Props Description

Button

label , variant , onClickAction

Clickable button

Critical Implementation Details

Action Object Structure

IMPORTANT: Use action.payload , NOT action.arguments :

WRONG - Will cause AttributeError

action.arguments

CORRECT

action.payload

Context Parameter

The context parameter is RequestContext , not dict :

WRONG - Tries to wrap RequestContext

request_context = RequestContext(metadata=context)

CORRECT - Use directly

user_id = context.user_id

UserMessageItem Required Fields

When creating synthetic user messages:

from chatkit.types import UserMessageItem, UserMessageTextContent

Include ALL required fields

synthetic_message = UserMessageItem( id=self.store.generate_item_id("message", thread, context), thread_id=thread.id, created_at=datetime.now(), content=[UserMessageTextContent(type="input_text", text=message_text)], inference_options={}, )

Anti-Patterns

• Mixing handlers - Don't handle same action in both client and server

• Missing payload - Always include data in action payload

• Using action.arguments - Use action.payload

• Wrapping RequestContext - Context is already RequestContext

• Missing UserMessageItem fields - Include id, thread_id, created_at

• Wrong content type - Use type="input_text" for user messages

Verification

Run: python3 scripts/verify.py

Expected: ✓ building-chat-widgets skill ready

If Verification Fails

• Check: references/ folder has widget-patterns.md

• Stop and report if still failing

References

• references/widget-patterns.md - Complete widget patterns

• references/server-action-handler.md - Backend action handling