Testing PRPM CLI

Overview

Test PRPM CLI commands against a local registry by building the package, setting the registry URL, and invoking the CLI directly. Includes comprehensive cross-format conversion testing patterns and contract testing to verify documented behavior matches implementation.

When to Use

• Testing new CLI commands or features

• Debugging CLI behavior

• Verifying CLI changes before committing

• Testing against local registry data

• Validating cross-format conversions

• Testing new format converters

• Contract testing: verifying documented behavior matches implementation

Quick Reference

Step Command

Build CLI npm run build --workspace=packages/cli

Start local registry npm run dev --workspace=packages/registry

Set local registry export PRPM_REGISTRY_URL=http://127.0.0.1:3111

Run CLI directly node /Users/khaliqgant/Projects/prpm/app/packages/cli/dist/index.js <command>

Run via npm link prpm <command> (after linking)

Workflow

1. Build the CLI (Required First Step)

Always rebuild before testing to ensure you're testing current code:

npm run build --workspace=packages/cli

2. Start Local Registry

Start the registry server in background:

npm run dev --workspace=packages/registry & sleep 3 lsof -i :3111 # Verify it's running

3. Configure Local Registry

Point CLI to local registry instead of production:

export PRPM_REGISTRY_URL=http://127.0.0.1:3111

Verify it's set:

echo $PRPM_REGISTRY_URL

4. Run CLI Commands

Option A: Direct invocation (recommended for testing)

node /Users/khaliqgant/Projects/prpm/app/packages/cli/dist/index.js search typescript node /Users/khaliqgant/Projects/prpm/app/packages/cli/dist/index.js install some-package

Option B: npm link (for interactive testing)

cd packages/cli npm link prpm search typescript

Comprehensive Conversion Testing

Use Self-Improving Skill to Find Test Packages

Before testing conversions, use the self-improving skill to download a diverse set of packages:

Search for packages of different types

node $CLI search "claude" --limit 10 node $CLI search "cursor" --limit 10 node $CLI search "agent" --limit 10 node $CLI search "skill" --limit 10

Create Test Directory and Install Diverse Packages

mkdir -p /tmp/prpm-conversion-tests cd /tmp/prpm-conversion-tests export PRPM_REGISTRY_URL=http://127.0.0.1:3111 CLI="/Users/khaliqgant/Projects/prpm/app/packages/cli/dist/index.js"

Install packages of different subtypes

node $CLI install @prpm/agent-builder-skill --as claude # Skill node $CLI install @prpm/creating-cursor-rules --as cursor # Cursor Rule node $CLI install @camoneart/context-engineering-agent --as claude # Agent

Supported Formats (CLI_SUPPORTED_FORMATS)

Test conversions across ALL supported formats:

Format Description

cursor Cursor IDE rules (.mdc)

claude Claude Code (skills, agents, commands)

windsurf Windsurf rules

continue Continue rules

copilot GitHub Copilot instructions

kiro Kiro steering files

agents.md Agents.md format

gemini Gemini CLI extensions

ruler Ruler format

zed Zed editor extensions

opencode OpenCode rules

aider Aider conventions

trae Trae rules

replit Replit agent rules

zencoder ZenCoder rules

droid Factory/Droid rules

Conversion Test Matrix

Run comprehensive conversion tests:

cd /tmp/prpm-conversion-tests mkdir -p /tmp/conversions CLI="/Users/khaliqgant/Projects/prpm/app/packages/cli/dist/index.js"

Claude Skill → All formats

for format in cursor windsurf kiro gemini zed continue copilot opencode aider trae replit zencoder droid; do node $CLI convert .claude/skills/*/SKILL.md --to $format -o /tmp/conversions/skill-to-$format.md 2>&1 done

Cursor Rule → Multiple formats

for format in claude windsurf gemini zed; do node $CLI convert .cursor/rules/*.mdc --to $format -o /tmp/conversions/cursor-to-$format.md 2>&1 done

Claude Agent → Multiple formats

for format in cursor gemini windsurf; do node $CLI convert .claude/agents/*.md --to $format -o /tmp/conversions/agent-to-$format.md 2>&1 done

Round-Trip Testing

Verify content preservation through round-trip conversions:

Claude → Cursor → Claude

node $CLI convert .claude/skills/*/SKILL.md --to cursor -o /tmp/conversions/step1-cursor.mdc node $CLI convert /tmp/conversions/step1-cursor.mdc --to claude -o /tmp/conversions/step2-claude.md

Compare file sizes (expect some reduction but not dramatic)

wc -c .claude/skills/*/SKILL.md /tmp/conversions/step1-cursor.mdc /tmp/conversions/step2-claude.md

Validation Checklist

For each conversion, verify:

• Command succeeds - Exit code 0, no errors

• Output file created - File exists at specified path

• Content preserved - Core markdown structure intact

• Format-specific frontmatter - Correct fields for target format

• File size reasonable - Not truncated (compare with source)

Expected File Sizes

Conversion Type Expected Size Ratio

Claude → Cursor ~95-100%

Claude → Windsurf ~95-100%

Claude → Gemini ~95-100%

Claude → OpenCode May be smaller (format limits)

Claude → Droid May be smaller (format limits)

Round-trip ~50-70% (metadata loss expected)

Test Report Template

Document results in this format:

Conversion Test Report

Test Setup

• Registry: Local (port 3111)
• Test packages: [list installed packages]

Results

Observations

• [Note any issues, truncations, or unexpected behavior]

Contract Testing (CRITICAL)

Contract testing ensures documented behavior matches implementation. This is the most important type of testing for features with configurable behavior.

Why Contract Testing Matters

The eager/lazy loading bug is a case study: documentation described a precedence chain (CLI > file > package > default), but implementation only handled CLI flags. Tests passed because they only tested the CLI flag path. Contract testing would have caught this.

Contract Testing Principles

• Test EVERY documented behavior path, not just the happy path

• Test precedence chains completely - if docs say "A > B > C > default", test all 4 cases

• Test behavior WITHOUT flags - verify defaults and configuration-driven behavior

• Test WITH and WITHOUT explicit settings - don't assume flag presence

Contract Testing Checklist

For ANY feature with documented behavior:

• Read the documentation first - what does it claim to do?

• List all behavior paths - every if/else/default mentioned

• Create test for each path - one test per documented behavior

• Test WITHOUT user flags - verify package/config defaults work

• Test precedence - verify higher-priority overrides lower

• Verify error messages match - documented errors should occur

Example: Eager/Lazy Loading Contract Tests

Documentation states: "Precedence: CLI flag > package-level > default (lazy)"

Required tests:

Setup test directory

mkdir -p /tmp/prpm-contract-tests cd /tmp/prpm-contract-tests export PRPM_REGISTRY_URL=http://127.0.0.1:3111 CLI="/Users/khaliqgant/Projects/prpm/app/packages/cli/dist/index.js"

Test 1: CLI --eager flag (highest priority)

rm -rf .openskills AGENTS.md node $CLI install @prpm/some-skill --as agents.md --eager

VERIFY: AGENTS.md contains activation="eager" or priority="0"

grep -q 'activation="eager"|priority="0"' AGENTS.md && echo "PASS: CLI --eager works" || echo "FAIL: CLI --eager"

Test 2: CLI --lazy flag overrides package eager

rm -rf .openskills AGENTS.md

Install a package that has eager:true in prpm.json with --lazy flag

node $CLI install @prpm/eager-package --as agents.md --lazy

VERIFY: Should be lazy despite package setting

grep -q 'activation="lazy"|priority="1"' AGENTS.md && echo "PASS: CLI --lazy overrides" || echo "FAIL: CLI --lazy"

Test 3: Package-level eager (NO CLI flag) - THIS IS THE BUG THAT WAS MISSED

rm -rf .openskills AGENTS.md

Install a package that has eager:true in its prpm.json WITHOUT --eager flag

node $CLI install @prpm/eager-package --as agents.md

VERIFY: Should be eager based on package setting

grep -q 'activation="eager"|priority="0"' AGENTS.md && echo "PASS: Package eager works" || echo "FAIL: Package eager"

Test 4: Default (lazy) when no flags and no package setting

rm -rf .openskills AGENTS.md node $CLI install @prpm/normal-skill --as agents.md

VERIFY: Should be lazy by default

grep -q 'activation="lazy"|priority="1"' AGENTS.md && echo "PASS: Default lazy works" || echo "FAIL: Default"

Contract Test Template

For any new feature, create tests in this format:

Contract Tests for [Feature Name]

Documentation Claims:

1. [Claim 1 from docs]
2. [Claim 2 from docs]
3. [Precedence/default behavior from docs]

Test Cases:

Results:

• All tests must pass before feature is considered complete
• Document any deviations between docs and implementation

Anti-Patterns to Avoid

Anti-Pattern Why It Fails Correct Approach

Only testing with flags Misses config/default paths Test without flags first

Assuming documentation is implementation Docs may describe intent, not reality Verify each claim with test

Testing happy path only Misses precedence bugs Test all documented paths

Skipping default behavior test Defaults often broken Always test "no config" case

Not reading docs before testing Miss documented behaviors Read docs, list claims, test each

Common Mistakes

Mistake Symptom Fix

Forgetting to build Old behavior, changes not reflected Run npm run build --workspace=packages/cli

Missing registry env Commands hit production registry Set PRPM_REGISTRY_URL before running

Stale npm link Wrong version running Re-run npm link after rebuilding

Local registry not running Connection refused errors Start registry: npm run dev --workspace=packages/registry

Testing single format only Miss format-specific bugs Test ALL formats in CLI_SUPPORTED_FORMATS

No round-trip testing Miss content loss bugs Always verify round-trip preservation

No contract testing Docs ≠ implementation Test EVERY documented behavior

Only testing with CLI flags Miss config/default bugs Test without flags to verify defaults

One-Liner Setup

npm run build --workspace=packages/cli && export PRPM_REGISTRY_URL=http://127.0.0.1:3111

Then test commands:

node packages/cli/dist/index.js --help

Unit Test Commands

Run converter unit tests:

All converter tests

npm run test --workspace=packages/converters

Specific test files

npm run test --workspace=packages/converters -- --testPathPattern="file-references" npm run test --workspace=packages/converters -- --testPathPattern="security" npm run test --workspace=packages/converters -- --testPathPattern="cross-format" npm run test --workspace=packages/converters -- --testPathPattern="zed"