Add E2E Spec
Create a new E2E_TESTS.md specification file for a package by analyzing its source code, commands, and public API.
Additional instructions from the user: "$ARGUMENTS". Ignore if empty.
Phase 1: Analyze Package
-
Identify the target package:
- If
$ARGUMENTSspecifies a package path, use that. - If in a package directory, use the current package.
- Otherwise, ask the user which package to analyze.
- If
-
Read the package to understand:
- What the package does (README, package manifest, main entry points)
- Public API / exported commands / CLI interface
- External dependencies (what it calls: Docker, HTTP APIs, CLI tools)
- Configuration files it reads or creates
- State it manages (databases, file stores, caches)
-
Check if an
E2E_TESTS.mdalready exists for this package. If it does, inform the user and suggest usingupdate-specinstead.
Phase 2: Draft Spec
-
Determine the suites. Each major command, workflow, or feature area becomes a suite.
-
Include an Index section at the top of the spec (after the title and description) with a markdown link to every suite (e.g.,
- [Suite Name](#suite-name)). -
For each suite, draft these sections using heading levels appropriate to their nesting depth (e.g., if the suite is an H2, then these sections are H3; if the suite is an H3, they are H4; and so on):
Preconditions(heading with bullet list): What setup is needed (temp dir, git repo, config files, mock tools)Features(heading) containing individual features as flat peer subheadings one level deeper. Do NOT create intermediate category headings (e.g.,#### Core,#### Edge) — categories are specified only via<!-- category: ... -->HTML comments on each feature. Aim for a mix of features across these categories:core: Primary happy-path scenariosedge: Boundary conditions (empty inputs, very long strings, special characters)error: Failure modes (missing config, invalid input, unavailable dependencies)side-effect: Observable interactions (hooks, notifications, file creation)idempotency: Safe repetition (re-running commands, re-creating resources)
Postconditions(heading with bullet list): Verifiable end states
-
For features requiring real external services, add
<!-- skip: ... -->metadata with clear documentation on how to unskip. -
Ensure every feature has:
- A
<!-- category: ... -->comment - At least one concrete, testable assertion bullet
- Present tense, declarative language
- A
Phase 3: Review
- Present the complete draft spec to the user.
- Show a summary:
- Number of suites
- Features per suite by category
- Total feature count
- Any features marked as skip
- Ask the user for feedback and adjustments.
Phase 4: Write
- Write the approved spec to
E2E_TESTS.mdin the package directory. - If a root-level
docs/E2E_TESTS.mdorE2E_TESTS.mddoesn't exist yet, offer to create one with project-wide testing constraints (isolation rules, cleanup patterns, environment variable conventions). - Inform the user they can now run:
review-specto validate the formatgenerateto create test files from the spec