Windows Desktop Automation with winguictl

Scripts

The skill includes a standalone CLI script:

• scripts\winguictl.py — Python CLI entry point (Windows only)

Quick start

# List windows (with window state, foreground flag, and hierarchical indentation)
python scripts\winguictl.py window list
# Control window state
python scripts\winguictl.py window --window-id <id> focus
python scripts\winguictl.py window --window-id <id> minimize
python scripts\winguictl.py window --window-id <id> maximize
python scripts\winguictl.py window --window-id <id> restore
python scripts\winguictl.py window --window-id <id> move --x 100 --y 200
python scripts\winguictl.py window --window-id <id> resize --width 800 --height 600
# Take a screenshot and save to file
python scripts\winguictl.py screenshot --window-id <id> --output artifacts\shot.png
# Get window structure snapshots
python scripts\winguictl.py snapshot --window-id <id> hwnd
python scripts\winguictl.py snapshot --window-id <id> uia
python scripts\winguictl.py snapshot --window-id <id> ocr

Commands

For detailed command documentation, see:

• Window - List all windows, control window state and position
• Snapshot - Get window structure snapshots
• Find - Find elements in a window
• Action - Execute interaction operations
• Control - Directly control specific controls (Win32 and UIA)
• Screenshot - Capture window screenshots
• Driver Test - Driver test steps

Workflow

1. List windows and identify the correct target — window list shows hierarchical parent-child relationships with indentation.
2. Prefer exact window ids over fuzzy titles when possible.
3. Use window focus to bring the target window to the foreground before interacting.
4. Use window minimize/maximize/restore/close/move/resize to control window state before interacting.
5. Use snapshot hwnd/uia/ocr to inspect window structure when locators are not obvious.
6. Prefer HWND and UIA locators over OCR and image matching — structured identifiers (hwnd, automation_id, runtime_id) are more reliable and deterministic than pixel-based approaches.

• For UIA controls, run snapshot uia first to get element automation_id or runtime_id, then use uia-control commands to interact.
• For Win32 controls, run snapshot hwnd to get control hwnd, then use control commands to interact.
• Use find ocr only for rendered text that is not exposed through UIA or window text.
• Use find image / click-image only for iconography, canvas content, or custom-painted controls where no structured locator exists.

11. Use relative window coordinates only when neither structured locators nor image matching are available.
12. Capture screenshots before or after important steps.
13. Return structured results, artifact paths, and any follow-up risk.

Operating Rules

• Coordinates are relative to the window unless the tool explicitly says otherwise.
• Use --dry-run when you need to preview coordinates or confirm intent.
• Report the exact window title and window_id you acted on.

Dependencies

Safety Boundary

• Use this skill for automation of the user's own software, test environments, or explicitly authorized systems.
• Do not use this skill to bypass third-party anti-bot checks, CAPTCHAs, or unrelated security controls.