LLM Council Skill

Council Protocol

Stage 1: Independent Responses

Dispatch the user's prompt to all available omega CLIs in parallel:

Check availability

node .claude/skills/omega-gemini-cli/scripts/verify-setup.mjs && HAS_GEMINI=1 node .claude/skills/omega-codex-cli/scripts/verify-setup.mjs && HAS_CODEX=1 node .claude/skills/omega-claude-cli/scripts/verify-setup.mjs && HAS_CLAUDE_CLI=1 node .claude/skills/omega-cursor-cli/scripts/verify-setup.mjs && HAS_CURSOR=1

Parallel dispatch to available CLIs

TMPDIR=$(mktemp -d) [ "$HAS_GEMINI" = "1" ] && node .claude/skills/omega-gemini-cli/scripts/ask-gemini.mjs "$PROMPT" --timeout-ms 120000 > "$TMPDIR/gemini.txt" 2>"$TMPDIR/gemini.err" & [ "$HAS_CODEX" = "1" ] && node .claude/skills/omega-codex-cli/scripts/ask-codex.mjs "$PROMPT" --timeout-ms 120000 > "$TMPDIR/codex.txt" 2>"$TMPDIR/codex.err" & [ "$HAS_CLAUDE_CLI" = "1" ] && node .claude/skills/omega-claude-cli/scripts/ask-claude.mjs "$PROMPT" --timeout-ms 120000 > "$TMPDIR/claude.txt" 2>"$TMPDIR/claude.err" & [ "$HAS_CURSOR" = "1" ] && node .claude/skills/omega-cursor-cli/scripts/ask-cursor.mjs "$PROMPT" --yolo --trust --timeout-ms 120000 > "$TMPDIR/cursor.txt" 2>"$TMPDIR/cursor.err" & wait

Stage 2: Anonymized Peer Review

• Collect all successful Stage 1 responses

• Assign anonymous labels: Response A, Response B, Response C, ...

• Maintain label_to_model mapping (e.g., A=gemini, B=codex, C=claude)

• Send all anonymized responses back to each available model with:

You are reviewing responses to this question: "$PROMPT"

Response A: [content]

Response B: [content]

Response C: [content]

Evaluate each response for accuracy, completeness, and reasoning quality. Then provide your FINAL RANKING (best to worst):

1. Response [X]
2. Response [Y]
3. Response [Z]

• Parse FINAL RANKING from each reviewer's output using regex: \d+.\s*Response [A-Z]

• Compute aggregate ranking (average position across all reviewers)

Stage 3: Chairman Synthesis

Send to chairman model (default: current Claude session or ask-claude.mjs):

You are the Chairman synthesizing a multi-model council discussion.

Original question: "$PROMPT"

Stage 1 Responses (with de-anonymized model names): [Model]: [Response] ...

Stage 2 Peer Review Rankings: Aggregate ranking: [best to worst with scores]

Synthesize the best insights from all responses into a single comprehensive answer. Highlight areas of consensus and dissent. Provide the strongest possible answer.

Usage

Full council (all available models)

Skill({ skill: 'llm-council' })

Then in agent: run full council protocol above

Quick consultation (skip peer review)

Stage 1 only -- parallel dispatch, collect responses, skip ranking

Use when speed matters more than rigorous evaluation

When to Use

• High-stakes architectural decisions requiring multiple perspectives

• Code review where diverse model viewpoints reduce blind spots

• Plan critique and validation

• Resolving disagreements about implementation approach

• Cross-validation of security analysis

• When the user explicitly requests "council", "multiple perspectives", or "cross-validate"

Iron Laws

• ALWAYS check CLI availability before dispatch -- never assume a model is present

• ALWAYS anonymize responses before peer review -- model identity bias is real

• NEVER skip Stage 2 for high-stakes decisions -- the peer review is the core innovation

• ALWAYS preserve all 3 stages in output for transparency

• ALWAYS set per-model timeout to prevent one slow model from blocking the council

Anti-Patterns

Anti-Pattern Why Bad Correct Approach

Dispatching to unavailable CLIs Silent failure, missing responses Run verify-setup.mjs first

Showing model names during peer review Introduces identity bias Use anonymous labels (Response A, B, C)

Using only 2 models for council Peer review meaningless with 2 Require minimum 3 for ranking value

Ignoring failed model responses May miss degradation Log failures, include in metadata

Running council for simple questions Massive overhead for trivial queries Reserve for high-stakes decisions

Exit Codes

Code Meaning

0 Council completed successfully (at least 2 models responded)

1 Council failed (fewer than 2 models available or all timed out)

Configuration

Env Var Default Purpose

LLM_COUNCIL_TIMEOUT_MS

120000 Per-model timeout for Stage 1 and Stage 2

LLM_COUNCIL_MIN_MODELS

2 Minimum models required for council to proceed

LLM_COUNCIL_CHAIRMAN

(current session) Chairman model for Stage 3 synthesis

Integration Notes

• This skill does NOT require the llm-council-master FastAPI server

• Uses omega wrapper scripts directly via Bash backgrounding for parallelism

• Anonymized peer review is preserved from llm-council-master's design (its core innovation)

• Chairman synthesis can use the current Claude session (no additional CLI call needed)

• Temporary files stored in system temp dir, cleaned up after council completes

• Minimum 2 available models required for council to proceed