Testing CLI Compatibility
Validates that the standalone Copilot CLI works correctly with this repo's skills and MCP servers.
Quick Start
Run all tests (including MCP)
./deployment/scripts/test-cli-compatibility.ps1
Skip MCP server tests (faster, no tool approval needed)
./deployment/scripts/test-cli-compatibility.ps1 -SkipMcp
What It Tests
Test What It Checks
1 CLI Version copilot command is installed and responds
2 Custom Instructions .github/copilot-instructions.md is loaded (identifies project name)
3 Skills All .github/skills/*/SKILL.md files are visible (count auto-detected)
4 MCP Servers playwright and microsoftdocs servers are connected
Prerequisites
-
Copilot CLI installed: npm install -g @anthropic-ai/copilot-cli or via GitHub
-
Authenticated: Run copilot login if not already logged in
-
MCP servers synced: Run the syncing-mcp-servers skill first if MCP tests fail
How It Works
The script uses copilot -p "<prompt>" -s --no-auto-update to invoke the CLI in non-interactive mode:
-
-p — Single prompt, exits after response
-
-s — Silent mode, output only the agent response (no stats)
-
--no-auto-update — Skip update check for faster execution
-
--allow-all-tools — Used only for MCP test (needs tool access)
Each test sends a prompt asking the CLI to list what it can see, then validates the response against expected values.
When to Run
-
After adding or modifying skills in .github/skills/
-
After syncing MCP servers with the syncing-mcp-servers skill
-
When a user reports CLI issues (missing agents, skills, or tools)
-
As a smoke test after CLI updates (copilot update )
Manual Testing
If the script isn't available, you can test manually:
Check version
copilot --version
Test custom instructions
copilot -p "What project is this repo for?" -s
Test skills
copilot -p "List all custom skills" -s
Test MCP (needs --allow-all-tools)
copilot -p "List all MCP servers" -s --allow-all-tools
Troubleshooting
Issue Fix
copilot not found Install CLI or add to PATH
Custom instructions not loaded Ensure you're in the repo directory
Agents missing No custom agents expected — this repo uses skills + copilot-instructions only
Skills missing Check .github/skills/*/SKILL.md files exist with valid frontmatter
MCP servers not connected Run syncing-mcp-servers skill to sync .vscode/mcp.json → ~/.copilot/mcp-config.json