a2a-role-auditor

Audits nodes/*.toml role templates in a tmux-a2a-postman project to fix node-to-node interaction breakdowns.

1. Mandatory Triage Gate

Before running any template audit, determine whether the issue is daemon-level or template-level.

Daemon-level indicators (stop here — report as config issue, do NOT produce patches):

• Wrong or missing edges in postman.toml

• nodes/{node}.toml file does not exist

• Session disabled in postman TUI

Template-level confirmed (node exists, edges correct, but behavior is wrong):

• Proceed to the 7-check audit below.

2. 7-Check Audit

2.1. Pre-check: File Existence (binary)

For every node referenced in postman.toml edges, verify nodes/{node}.toml exists.

• PASS: file present

• FAIL: file missing → emit BLOCKING finding; abort all further checks for that node

2.2. Check 1 — PONG Awareness

The daemon calls MarkPongReceived() only when info.To == "postman" (message.go). A node that does not send TO postman as the explicit recipient is never marked PONG-active and remains invisible in other nodes' talks_to_line .

• PASS: template explicitly instructs the node to send a message with "postman" as recipient (e.g., "Send a PONG addressed to postman")

• FAIL: template references postman only as a routing mechanism ("via postman"), or has no postman mention at all

2.3. Check 2 — Routing Clarity

• PASS: template names at least one recipient for output messages

• FAIL: template says "send a message" without specifying who receives it

2.4. Check 3 — Completion Protocol

• PASS: template specifies a machine-readable signal word (e.g., APPROVED, DONE, BLOCKED) for task completion

• FAIL: completion state is undefined or described only in natural language

2.5. Check 4 — Fallback Routing

• PASS: template names an alternative recipient when the primary contact is absent from talks_to_line

• FAIL: no fallback specified

2.6. Check 5 — Cross-Edge Consistency

Two sub-checks:

• Binary: does the template mention only nodes that exist as edges in postman.toml ? (PASS/FAIL — no judgment)

• Judgment: are the described routing semantics consistent with edge direction? (LLM assessment — label findings with Type: JUDGMENT-BASED )

2.7. Check 6 — on_join Completeness

• PASS: on_join field is non-empty

• FAIL: on_join = ""

3. Findings Format

Every finding MUST use this exact schema:

[SEVERITY] Node: {node} Field: nodes/{node}.toml:[{node}].{field} Check: {check name} [Type: JUDGMENT-BASED] <- optional, only when applicable Result: FAIL Issue: {description} Fix: {exact replacement text}

Severity: BLOCKING | IMPORTANT | MINOR

Type: JUDGMENT-BASED is a separate flag, not a severity level. Present findings in order: BLOCKING first, then IMPORTANT, then MINOR.

4. Workflow

• Read postman.toml — extract edges, build adjacency map

• Read each nodes/{node}.toml (source of truth; runtime session templates are NOT compared)

• For each node: run Pre-check, then Checks 1–6 in order

• Produce findings report sorted by severity

• Propose concrete patch text for every finding

• Present to user for feedback; iterate until approved

NOTE: Do NOT auto-apply patches. Propose only; the user applies manually or delegates to worker.

5. Baseline Examples

The following issues were identified in a real audit session and serve as illustrative examples.

5.1. Example 1 — Routing clarity (IMPORTANT)

[IMPORTANT] Node: critic Field: nodes/critic.toml:[critic].template Check: Routing clarity Result: FAIL Issue: Template says "send a message via postman" without specifying a recipient. Fix: "Send findings to orchestrator. If orchestrator is absent from talks_to_line, send to guardian who will relay."

5.2. Example 2 — Completion protocol (IMPORTANT)

[IMPORTANT] Node: boss Field: nodes/boss.toml:[boss].template Check: Completion protocol Result: FAIL Issue: No machine-readable approval signal defined. Recipients cannot parse the response programmatically. Fix: "Reply with 'APPROVED: <summary>' when approving, or 'REJECTED: <reason>' when rejecting."

5.3. Example 3 — Cross-edge consistency (IMPORTANT, JUDGMENT-BASED)

[IMPORTANT] Node: guardian Field: nodes/guardian.toml:[guardian].template Check: Cross-edge consistency Type: JUDGMENT-BASED Result: FAIL Issue: Guardian acts as a routing relay between critic and orchestrator, but this is not documented. Forwarding-to-boss path is also absent. Fix: "After receiving critic findings: if approved, forward to boss. If rejected, return to critic with specific revision request."

5.4. Example 4 — on_join completeness (MINOR)

[MINOR] Node: worker Field: nodes/worker.toml:[worker].on_join Check: on_join completeness Result: FAIL Issue: on_join is empty; node receives no startup context. Fix: on_join = "You are worker. Send PONG to postman on startup, then await task assignment from orchestrator."

5.5. Example 5 — PONG awareness (BLOCKING)

[BLOCKING] Node: orchestrator Field: nodes/orchestrator.toml:[orchestrator].template Check: PONG awareness Result: FAIL Issue: Template references postman only as a routing mechanism, never as explicit recipient. Node will not be marked PONG-active and will be invisible in other nodes' talks_to_line. Fix: Add to startup section: "On session start, send a PONG message addressed TO postman (recipient = postman, not via postman). This registers you as active."

6. Constraints

• Propose patches only; do NOT auto-apply

• When an issue is daemon-level (wrong edges, missing file, disabled session), note it as a config finding — template patches cannot fix it

• Manual integration test for talks_to_line visibility: if both nodes have not yet sent PONG in the current session, mark the result INCONCLUSIVE (environment), not a skill failure