Azure DevOps Resource Validator Skill

Purpose: Validate and auto-create Azure DevOps projects and resources, ensuring .env configuration is correct.

Auto-Activation: Triggers when Azure DevOps setup or validation is needed.

What This Skill Does

This skill ensures your Azure DevOps configuration in .env is valid and all resources exist. It's smart enough to:

• Validate Azure DevOps projects - Check if projects exist (multiple for project-per-team)

• Prompt for action - Select existing project or create new one

• Validate area paths - Check if area paths exist (for area-path-based strategy)

• Create missing area paths - Auto-create area paths if missing

• Validate teams - Check if teams exist (for team-based strategy)

• Update .env with correct values - Ensure configuration is valid

When This Skill Activates

✅ Automatically activates when:

• You set up Azure DevOps integration for the first time

• You run /sw-ado:sync and resources are missing

• Your .env has invalid Azure DevOps configuration

• You mention "ado setup" or "azure devops validation"

Azure DevOps Configuration Structure

Required .env Variables

AZURE_DEVOPS_PAT=your_token_here AZURE_DEVOPS_ORG=yourorganization AZURE_DEVOPS_STRATEGY=project-per-team # or area-path-based, team-based

Strategy-Specific Variables

Strategy 1: Project-per-team (Multiple Projects)

AZURE_DEVOPS_STRATEGY=project-per-team AZURE_DEVOPS_PROJECTS=WebApp,MobileApp,Platform

→ Validates that WebApp, MobileApp, and Platform projects exist

Strategy 2: Area-path-based (One Project, Multiple Area Paths)

AZURE_DEVOPS_STRATEGY=area-path-based AZURE_DEVOPS_PROJECT=MainProduct AZURE_DEVOPS_AREA_PATHS=Frontend,Backend,Mobile

→ Validates MainProduct project exists → Creates area paths if missing: MainProduct\Frontend, MainProduct\Backend, MainProduct\Mobile

Strategy 3: Team-based (One Project, Multiple Teams)

AZURE_DEVOPS_STRATEGY=team-based AZURE_DEVOPS_PROJECT=MainProduct AZURE_DEVOPS_TEAMS=Alpha Team,Beta Team,Gamma Team

→ Validates MainProduct project exists → Creates teams if missing: Alpha Team, Beta Team, Gamma Team

NEW: Per-Project Configuration (Advanced - Multiple Projects × Resources)

Multiple projects with their own area paths and teams

AZURE_DEVOPS_STRATEGY=project-per-team AZURE_DEVOPS_PROJECTS=Backend,Frontend,Mobile

Per-project area paths (hierarchical naming)

AZURE_DEVOPS_AREA_PATHS_Backend=API,Database,Cache AZURE_DEVOPS_AREA_PATHS_Frontend=Web,Admin,Public AZURE_DEVOPS_AREA_PATHS_Mobile=iOS,Android,Shared

Per-project teams (optional)

AZURE_DEVOPS_TEAMS_Backend=Alpha,Beta AZURE_DEVOPS_TEAMS_Frontend=Gamma

→ Validates 3 projects exist: Backend, Frontend, Mobile → Creates area paths per project:

• Backend\API, Backend\Database, Backend\Cache

• Frontend\Web, Frontend\Admin, Frontend\Public

• Mobile\iOS, Mobile\Android, Mobile\Shared → Creates teams per project:

• Backend: Alpha, Beta

• Frontend: Gamma

Naming Convention: {PROVIDER}{RESOURCE_TYPE}{PROJECT_NAME}

Validation Flow

Step 1: Strategy Detection

Read .env and detect strategy:

AZURE_DEVOPS_STRATEGY=project-per-team

Result:

🔍 Detected strategy: Project-per-team Projects to validate: WebApp, MobileApp, Platform

Step 2: Project Validation (Project-per-team)

Check if projects exist:

API calls to Azure DevOps

GET https://dev.azure.com/{org}/_apis/projects/WebApp GET https://dev.azure.com/{org}/_apis/projects/MobileApp GET https://dev.azure.com/{org}/_apis/projects/Platform

If all projects exist:

✅ All projects validated: • WebApp (ID: abcd1234) • MobileApp (ID: efgh5678) • Platform (ID: ijkl9012)

If some projects don't exist:

⚠️ Projects not found: ✅ WebApp (exists) ❌ MobileApp (not found) ❌ Platform (not found)

What would you like to do?

1. Create missing projects
2. Select existing projects
3. Fix project names manually
4. Cancel

Your choice [1]:

Option 1: Create Missing Projects:

📦 Creating Azure DevOps projects...

Creating project: MobileApp... ✅ Project created: MobileApp (ID: mnop3456)

Creating project: Platform... ✅ Project created: Platform (ID: qrst7890)

✅ All projects now exist!

Option 2: Select Existing Projects:

Available projects in organization:

1. WebApp
2. ApiGateway
3. AuthService
4. NotificationService
5. DataPipeline

Select projects (comma-separated numbers) [2,3]:

✅ Updated .env: AZURE_DEVOPS_PROJECTS=WebApp,ApiGateway,AuthService

Step 3: Area Path Validation (Area-path-based)

Scenario: One project with area paths

AZURE_DEVOPS_STRATEGY=area-path-based AZURE_DEVOPS_PROJECT=MainProduct AZURE_DEVOPS_AREA_PATHS=Frontend,Backend,Mobile,QA

Validation:

Checking project: MainProduct... ✅ Project "MainProduct" exists

Checking area paths... ✅ MainProduct\Frontend (exists) ✅ MainProduct\Backend (exists) ⚠️ MainProduct\Mobile (not found) ⚠️ MainProduct\QA (not found)

📦 Creating missing area paths... ✅ Created: MainProduct\Mobile ✅ Created: MainProduct\QA

✅ All area paths validated/created successfully

Step 4: Team Validation (Team-based)

Scenario: One project with multiple teams

AZURE_DEVOPS_STRATEGY=team-based AZURE_DEVOPS_PROJECT=MainProduct AZURE_DEVOPS_TEAMS=Alpha Team,Beta Team,Gamma Team

Validation:

Checking project: MainProduct... ✅ Project "MainProduct" exists

Checking teams... ✅ Alpha Team (exists) ⚠️ Beta Team (not found) ⚠️ Gamma Team (not found)

📦 Creating missing teams... ✅ Created: Beta Team ✅ Created: Gamma Team

✅ All teams validated/created successfully

Usage Examples

Example 1: Fresh Azure DevOps Setup (Project-per-team)

Scenario: New setup with multiple projects for different teams

Action: Run /sw-ado:sync

What Happens:

🔍 Validating Azure DevOps configuration...

Strategy: Project-per-team Checking projects: WebApp, MobileApp, Platform...

⚠️ Projects not found: • WebApp • MobileApp • Platform

What would you like to do?

1. Create new projects
2. Select existing projects
3. Cancel

Your choice [1]: 1

📦 Creating Azure DevOps projects...

Creating project: WebApp Description: Web application frontend Process template: Agile ✅ Created: WebApp (ID: proj-001)

Creating project: MobileApp Description: Mobile application Process template: Agile ✅ Created: MobileApp (ID: proj-002)

Creating project: Platform Description: Backend platform services Process template: Agile ✅ Created: Platform (ID: proj-003)

🎉 Azure DevOps configuration complete! All resources ready.

Example 2: Migrate from Single to Multi-Project

Scenario: Currently using single project, want to split into multiple

Current .env:

AZURE_DEVOPS_PROJECT=MainProduct

New .env:

AZURE_DEVOPS_STRATEGY=project-per-team AZURE_DEVOPS_PROJECTS=MainProduct-Frontend,MainProduct-Backend,MainProduct-Mobile

What Happens:

🔍 Detected strategy change: team-based → project-per-team

Validating new projects... ✅ MainProduct-Frontend (exists from previous split) ⚠️ MainProduct-Backend (not found) ⚠️ MainProduct-Mobile (not found)

Would you like to:

1. Create missing projects
2. Keep single project with area paths instead
3. Cancel

Your choice [1]: 1

📦 Creating projects... ✅ Created: MainProduct-Backend ✅ Created: MainProduct-Mobile

💡 Tip: You can now organize specs by project: .specweave/docs/internal/specs/MainProduct-Frontend/ .specweave/docs/internal/specs/MainProduct-Backend/ .specweave/docs/internal/specs/MainProduct-Mobile/

Example 3: Area Path Setup

Scenario: Large monolithic project with area-based organization

Action: Setup area paths for team organization

What Happens:

🔍 Validating Azure DevOps configuration...

Strategy: Area-path-based Project: EnterpriseApp Area Paths: Core, UserManagement, Billing, Reports, Analytics

Checking project: EnterpriseApp... ✅ Project exists

Checking area paths... ✅ EnterpriseApp\Core ✅ EnterpriseApp\UserManagement ⚠️ EnterpriseApp\Billing (not found) ⚠️ EnterpriseApp\Reports (not found) ⚠️ EnterpriseApp\Analytics (not found)

📦 Creating area paths...

Creating: EnterpriseApp\Billing ✅ Area path created with default team

Creating: EnterpriseApp\Reports ✅ Area path created with default team

Creating: EnterpriseApp\Analytics ✅ Area path created with default team

✅ All area paths ready!

Work items will be organized by area: • Billing features → EnterpriseApp\Billing • Report features → EnterpriseApp\Reports • Analytics features → EnterpriseApp\Analytics

Implementation Details

Location: src/utils/external-resource-validator.ts

Core Classes:

// Main validator class export class AzureDevOpsResourceValidator { private pat: string; private organization: string; private envPath: string;

constructor(envPath: string = '.env') { this.envPath = envPath; const env = this.loadEnv(); this.pat = env.AZURE_DEVOPS_PAT || ''; this.organization = env.AZURE_DEVOPS_ORG || ''; }

// Main validation entry point async validate(): Promise<AzureDevOpsValidationResult> { const env = this.loadEnv(); const strategy = env.AZURE_DEVOPS_STRATEGY || 'project-per-team';

// Validate based on strategy
if (strategy === 'project-per-team') {
  return this.validateMultipleProjects(projectNames);
} else if (strategy === 'area-path-based') {
  return this.validateAreaPaths(projectName, areaPaths);
} else if (strategy === 'team-based') {
  return this.validateTeams(projectName, teams);
}

} }

// Public API function export async function validateAzureDevOpsResources( envPath: string = '.env' ): Promise<AzureDevOpsValidationResult> { const validator = new AzureDevOpsResourceValidator(envPath); return validator.validate(); }

Key Implementation Features:

Async Project Creation (ADO-specific):

// ADO creates projects asynchronously - need to poll for completion async createProject(name: string): Promise<AzureDevOpsProject> { const result = await this.callAzureDevOpsApi('projects?api-version=7.0', 'POST', body);

// Wait for project to be fully created (ADO async behavior) await this.waitForProjectCreation(result.id);

return { id: result.id, name, description }; }

// Poll until project is in 'wellFormed' state private async waitForProjectCreation(projectId: string): Promise<void> { const maxAttempts = 30; // 30 seconds max wait for (let i = 0; i < maxAttempts; i++) { const project = await this.getProject(projectId); if (project.state === 'wellFormed') { return; // Project is ready! } await new Promise(resolve => setTimeout(resolve, 1000)); // Wait 1 second } throw new Error('Project creation timeout'); }

Interactive Prompts (when resources missing):

const { action } = await inquirer.prompt([ { type: 'select', name: 'action', message: Project "${projectName}" not found. What would you like to do?, choices: [ { name: 'Create new project', value: 'create' }, { name: 'Select existing project', value: 'select' }, { name: 'Skip this project', value: 'skip' }, { name: 'Cancel', value: 'cancel' } ] } ]);

Automatic .env Updates:

// After creating projects, update .env updateEnv(key: string, value: string): void { const envContent = fs.readFileSync(this.envPath, 'utf-8'); const updated = envContent.replace( new RegExp(^${key}=.*$, 'm'), ${key}=${value} ); fs.writeFileSync(this.envPath, updated); }

CLI Command

Automatic validation (during setup):

Runs automatically during specweave init

npx specweave init

Also runs automatically before sync

/sw-ado:sync 0014

Manual validation:

Via skill activation

"Can you validate my Azure DevOps configuration?"

Via TypeScript directly

npx tsx -e "import { validateAzureDevOpsResources } from './dist/utils/external-resource-validator.js'; await validateAzureDevOpsResources();"

Via CLI (future command - planned)

specweave validate-ado

Validation output:

interface AzureDevOpsValidationResult { valid: boolean; strategy: 'project-per-team' | 'area-path-based' | 'team-based'; projects: Array<{ name: string; id: string; exists: boolean; }>; created: string[]; // Names of newly created resources envUpdated: boolean; // Whether .env was modified }

// Example output: { valid: true, strategy: 'project-per-team', projects: [ { name: 'WebApp', id: 'proj-001', exists: true }, { name: 'MobileApp', id: 'proj-002', exists: true, created: true }, { name: 'Platform', id: 'proj-003', exists: true, created: true } ], created: ['MobileApp', 'Platform'], envUpdated: false }

Smart Project Detection

Auto-detect Based on Work Item Patterns

The skill can intelligently suggest project organization based on your existing work items:

// Analyze existing work items const workItems = await analyzeWorkItems(org, project);

// Detect patterns const patterns = { byArea: workItems.groupBy('areaPath'), // Area-based organization byTeam: workItems.groupBy('assignedTeam'), // Team-based organization byType: workItems.groupBy('workItemType') // Type-based organization };

// Suggest strategy if (patterns.byArea.length > 3) { console.log('💡 Detected area-based organization'); console.log(' Suggested strategy: area-path-based'); } else if (patterns.byTeam.length > 2) { console.log('💡 Detected team-based organization'); console.log(' Suggested strategy: team-based or project-per-team'); }

Project Creation API

Azure DevOps REST API (v7.0):

Create Project

POST https://dev.azure.com/{org}/_apis/projects?api-version=7.0 Content-Type: application/json Authorization: Basic {base64(":PAT")}

{ "name": "MobileApp", "description": "Mobile application project", "capabilities": { "versioncontrol": { "sourceControlType": "Git" }, "processTemplate": { "templateTypeId": "adcc42ab-9882-485e-a3ed-7678f01f66bc" # Agile } } }

Response: { "id": "proj-002", "name": "MobileApp", "state": "wellFormed" }

Create Area Path

POST https://dev.azure.com/{org}/{project}/_apis/wit/classificationnodes/areas?api-version=7.0 Content-Type: application/json

{ "name": "Frontend", "attributes": { "startDate": null, "finishDate": null } }

Response: { "id": 123, "name": "Frontend", "path": "\MainProduct\Area\Frontend" }

Create Team

POST https://dev.azure.com/{org}/_apis/projects/{projectId}/teams?api-version=7.0 Content-Type: application/json

{ "name": "Alpha Team", "description": "Alpha development team" }

Response: { "id": "team-001", "name": "Alpha Team", "projectName": "MainProduct" }

Configuration Examples

Example 1: Microservices Architecture (Project-per-team)

Before (.env ):

AZURE_DEVOPS_ORG=mycompany AZURE_DEVOPS_PAT=xxx

After validation:

AZURE_DEVOPS_ORG=mycompany AZURE_DEVOPS_PAT=xxx AZURE_DEVOPS_STRATEGY=project-per-team AZURE_DEVOPS_PROJECTS=AuthService,UserService,PaymentService,NotificationService

Folder structure created:

.specweave/docs/internal/specs/ ├── AuthService/ │ └── spec-001-oauth-implementation.md ├── UserService/ │ └── spec-001-user-management.md ├── PaymentService/ │ └── spec-001-stripe-integration.md └── NotificationService/ └── spec-001-email-notifications.md

Example 2: Monolithic Application (Area-path-based)

Before (.env ):

AZURE_DEVOPS_PROJECT=ERP

After validation:

AZURE_DEVOPS_ORG=enterprise AZURE_DEVOPS_PAT=xxx AZURE_DEVOPS_STRATEGY=area-path-based AZURE_DEVOPS_PROJECT=ERP AZURE_DEVOPS_AREA_PATHS=Finance,HR,Inventory,Sales,Reports

Work item organization:

ERP ├── Finance/ → Finance module features ├── HR/ → HR module features ├── Inventory/ → Inventory management ├── Sales/ → Sales module features └── Reports/ → Reporting features

Example 3: Platform Teams (Team-based)

Before (.env ):

AZURE_DEVOPS_PROJECT=Platform

After validation:

AZURE_DEVOPS_ORG=techcorp AZURE_DEVOPS_PAT=xxx AZURE_DEVOPS_STRATEGY=team-based AZURE_DEVOPS_PROJECT=Platform AZURE_DEVOPS_TEAMS=Infrastructure,Security,Data,DevOps

Team assignments:

• Infrastructure Team → Cloud resources, networking

• Security Team → Auth, compliance, auditing

• Data Team → Databases, analytics, ML

• DevOps Team → CI/CD, monitoring, tooling

Error Handling

Error 1: Invalid Credentials

Symptom: API calls fail with 401 Unauthorized

Solution:

❌ Azure DevOps API authentication failed

Please check:

1. AZURE_DEVOPS_PAT is correct
2. Token has not expired
3. AZURE_DEVOPS_ORG is correct

Generate new token at: https://dev.azure.com/{org}/_usersSettings/tokens

Error 2: Insufficient Permissions

Symptom: Cannot create projects (403 Forbidden)

Solution:

❌ Insufficient permissions to create projects

You need:

• Project Collection Administrator role (for creating projects)
• Project Administrator role (for area paths and teams)

Contact your Azure DevOps administrator to request permissions.

Error 3: Project Name Conflicts

Symptom: Project creation fails (name exists)

Solution:

❌ Project name "WebApp" already exists

Options:

1. Use a different project name
2. Select the existing project
3. Add a suffix (e.g., WebApp-v2)

Your choice [2]:

Error 4: Organization Limits

Symptom: Cannot create more projects

Solution:

❌ Organization project limit reached (250 projects)

Consider:

1. Using area-path-based strategy (one project)
2. Archiving old projects
3. Upgrading organization plan

Contact Azure DevOps support for limit increases.

Integration with SpecWeave Workflow

Automatic Validation

When using /sw-ado:sync , validation runs automatically:

/sw-ado:sync 0014

Internally calls:

1. validateAzureDevOpsResources()
2. Fix missing projects/area paths/teams
3. Create folder structure for specs
4. Proceed with sync

Manual Validation

Run validation independently:

Via skill

"Validate my Azure DevOps configuration"

Via TypeScript

npx tsx src/utils/external-resource-validator.ts --provider=ado

Via CLI (future)

specweave validate-ado

Best Practices

✅ Choose the right strategy:

• Project-per-team: Best for autonomous teams, microservices

• Area-path-based: Best for monolithic apps, shared codebase

• Team-based: Best for small organizations, simple structure

✅ Use descriptive names:

Good

AZURE_DEVOPS_PROJECTS=UserManagement,PaymentProcessing,NotificationEngine

Bad

AZURE_DEVOPS_PROJECTS=Proj1,Proj2,Proj3

✅ Document project mapping (in README):

Azure DevOps Projects

• UserManagement: User authentication and profile management
• PaymentProcessing: Payment gateway integrations
• NotificationEngine: Email, SMS, and push notifications

✅ Keep .env in version control (gitignored tokens):

Commit project structure

AZURE_DEVOPS_STRATEGY=project-per-team AZURE_DEVOPS_PROJECTS=WebApp,MobileApp,Platform

Don't commit sensitive data

AZURE_DEVOPS_PAT=<redacted>

Folder Organization

Based on strategy, the skill creates appropriate folder structure:

Project-per-team Structure

.specweave/docs/internal/specs/ ├── WebApp/ │ ├── spec-001-user-interface.md │ └── spec-002-responsive-design.md ├── MobileApp/ │ ├── spec-001-ios-features.md │ └── spec-002-android-features.md └── Platform/ ├── spec-001-api-design.md └── spec-002-database-schema.md

Area-path-based Structure

.specweave/docs/internal/specs/MainProduct/ ├── Frontend/ │ └── spec-001-ui-components.md ├── Backend/ │ └── spec-001-api-endpoints.md └── Mobile/ └── spec-001-mobile-sync.md

Team-based Structure

.specweave/docs/internal/specs/MainProduct/ ├── AlphaTeam/ │ └── spec-001-feature-a.md ├── BetaTeam/ │ └── spec-001-feature-b.md └── GammaTeam/ └── spec-001-feature-c.md

Key Differences from JIRA

Azure DevOps vs JIRA Resource Creation:

Aspect Azure DevOps JIRA

Project Creation Asynchronous (polling required) Synchronous (immediate)

Creation Time 5-30 seconds <1 second

Status Tracking Poll state field ('wellFormed') No polling needed

API Complexity Higher (async handling) Lower (sync operations)

Board Creation Auto-created with project Requires separate API call

Process Templates Required (Agile, Scrum, CMMI) Not applicable

Why Async Matters:

When you create an ADO project, the API returns immediately with state: 'new' , but the project isn't usable yet. The validator polls every 1 second (max 30 attempts) until state: 'wellFormed' :

// Create project (returns immediately) const project = await createProject('MobileApp'); // state: 'new'

// Poll until ready await waitForProjectCreation(project.id); // Polls until state: 'wellFormed'

// Now safe to use! console.log('✅ Project ready for work items');

Impact on UX:

• JIRA: "✅ Project created" (instant)

• ADO: "📦 Creating project... ⏳ Waiting for Azure DevOps to complete setup... ✅ Project ready!" (5-30s)

Summary

This skill ensures your Azure DevOps configuration is always valid by:

• ✅ Validating projects - Check if projects exist, prompt to select or create

• ✅ Supporting multiple strategies - Project-per-team, area-path-based, team-based

• ✅ Auto-creating resources - Projects, area paths, teams (with async handling)

• ✅ Organizing specs - Create folder structure based on projects

• ✅ Clear error messages - Actionable guidance for all failures

• ✅ Handles ADO async behavior - Polls for project creation completion

Result: Zero manual Azure DevOps setup - system handles everything, including ADO's async project creation!

Skill Version: 1.1.0 Introduced: SpecWeave v0.17.0 Last Updated: 2025-11-11 Key Changes v1.1.0: Added implementation details, async project creation handling, JIRA comparison