umbraco-e2e-testing

End-to-end testing for Umbraco backoffice extensions using Playwright and @umbraco/playwright-testhelpers . This approach tests against a real running Umbraco instance, validating complete user workflows.

Safety Notice

This listing is imported from skills.sh public index metadata. Review upstream SKILL.md and repository scripts before running.

Copy this and send it to your AI assistant to learn

Install skill "umbraco-e2e-testing" with this command: npx skills add umbraco/umbraco-cms-backoffice-skills/umbraco-umbraco-cms-backoffice-skills-umbraco-e2e-testing

Umbraco E2E Testing

End-to-end testing for Umbraco backoffice extensions using Playwright and @umbraco/playwright-testhelpers . This approach tests against a real running Umbraco instance, validating complete user workflows.

Critical: Use Testhelpers for Core Umbraco

Use @umbraco/playwright-testhelpers for core Umbraco operations:

Package Purpose Why Required

@umbraco/playwright-testhelpers

UI and API helpers Handles auth, navigation, core entity CRUD

@umbraco/json-models-builders

Test data builders Creates valid Umbraco entities with correct structure

Why use testhelpers for core Umbraco?

  • Umbraco uses data-mark instead of data-testid

  • testhelpers handle this

  • Auth token management is complex - testhelpers manage STORAGE_STAGE_PATH

  • API setup/teardown requires specific payload formats - builders ensure correctness

  • Selectors change between versions - testhelpers abstract these away

// WRONG - Raw Playwright for core Umbraco (brittle) await page.goto('/umbraco'); await page.fill('[name="email"]', 'admin@example.com');

// CORRECT - Testhelpers for core Umbraco import { test } from '@umbraco/playwright-testhelpers';

test('my test', async ({ umbracoApi, umbracoUi }) => { await umbracoUi.goToBackOffice(); await umbracoUi.login.enterEmail('admin@example.com'); });

When to Use Raw Playwright

For custom extensions, use umbracoUi.page (raw Playwright) because testhelpers don't know about your custom elements:

test('my custom extension', async ({ umbracoUi }) => { // Testhelpers for core navigation await umbracoUi.goToBackOffice(); await umbracoUi.content.goToSection(ConstantHelper.sections.settings);

// Raw Playwright for YOUR custom elements await umbracoUi.page.getByRole('link', { name: 'My Custom Item' }).click(); await expect(umbracoUi.page.locator('my-custom-workspace')).toBeVisible(); });

Use Testhelpers For Use umbracoUi.page For

Login/logout Custom tree items

Navigate to ANY section (including custom) Custom workspace elements

Create/edit documents via API Custom entity actions

Built-in UI interactions Custom UI components

When to Use

  • Testing complete user workflows

  • Testing data persistence

  • Testing authentication/authorization

  • Acceptance testing before release

  • Integration testing with real API responses

Related Skills

  • umbraco-testing - Master skill for testing overview

  • umbraco-playwright-testhelpers - Full reference for the testhelpers package

  • umbraco-test-builders - JsonModels.Builders for test data

  • umbraco-mocked-backoffice - Test without real backend (faster)

Documentation

Setup

Dependencies

Add to package.json :

{ "devDependencies": { "@playwright/test": "^1.56", "@umbraco/playwright-testhelpers": "^17.0.15", "@umbraco/json-models-builders": "^2.0.42", "dotenv": "^16.3.1" }, "scripts": { "test:e2e": "playwright test", "test:e2e:ui": "playwright test --ui", "test:e2e:debug": "playwright test --debug" } }

Then run:

npm install npx playwright install chromium

Version Compatibility: Match testhelpers to your Umbraco version:

Umbraco Testhelpers

17.1.x (pre-release) 17.1.0-beta.x

17.0.x ^17.0.15

14.x ^14.x

Configuration

Create playwright.config.ts :

import { defineConfig, devices } from '@playwright/test'; import { fileURLToPath } from 'url'; import { dirname, join } from 'path';

const __filename = fileURLToPath(import.meta.url); const __dirname = dirname(__filename);

export const STORAGE_STATE = join(__dirname, 'tests/e2e/.auth/user.json');

// CRITICAL: Testhelpers read auth tokens from this file process.env.STORAGE_STAGE_PATH = STORAGE_STATE;

export default defineConfig({ testDir: './tests/e2e', timeout: 30 * 1000, expect: { timeout: 5000 }, fullyParallel: false, forbidOnly: !!process.env.CI, retries: process.env.CI ? 2 : 0, workers: 1, reporter: process.env.CI ? 'line' : 'html', use: { baseURL: process.env.UMBRACO_URL || 'https://localhost:44325', trace: 'retain-on-failure', ignoreHTTPSErrors: true, // CRITICAL: Umbraco uses 'data-mark' not 'data-testid' testIdAttribute: 'data-mark', }, projects: [ { name: 'setup', testMatch: '/*.setup.ts', }, { name: 'e2e', testMatch: '/*.spec.ts', dependencies: ['setup'], use: { ...devices['Desktop Chrome'], ignoreHTTPSErrors: true, storageState: STORAGE_STATE, }, }, ], });

Critical Settings

Setting Value Why Required

testIdAttribute

'data-mark'

Umbraco uses data-mark , not data-testid

STORAGE_STAGE_PATH

Path to user.json Testhelpers read auth tokens from this file

ignoreHTTPSErrors

true

For local dev with self-signed certs

Without testIdAttribute: 'data-mark' , all getByTestId() calls will fail.

Authentication Setup

Create tests/e2e/auth.setup.ts :

import { test as setup } from '@playwright/test'; import { STORAGE_STATE } from '../../playwright.config'; import { ConstantHelper, UiHelpers } from '@umbraco/playwright-testhelpers';

setup('authenticate', async ({ page }) => { const umbracoUi = new UiHelpers(page);

await umbracoUi.goToBackOffice(); await umbracoUi.login.enterEmail(process.env.UMBRACO_USER_LOGIN!); await umbracoUi.login.enterPassword(process.env.UMBRACO_USER_PASSWORD!); await umbracoUi.login.clickLoginButton(); await umbracoUi.login.goToSection(ConstantHelper.sections.settings); await page.context().storageState({ path: STORAGE_STATE }); });

Environment Variables

Create .env (add to .gitignore ):

UMBRACO_URL=https://localhost:44325 UMBRACO_USER_LOGIN=admin@example.com UMBRACO_USER_PASSWORD=yourpassword UMBRACO_DATA_PATH=/path/to/Umbraco.Web.UI/App_Data # Optional: for data reset

Variable Required Purpose

UMBRACO_URL

Yes Backoffice URL

UMBRACO_USER_LOGIN

Yes Admin email

UMBRACO_USER_PASSWORD

Yes Admin password

UMBRACO_DATA_PATH

No App_Data path for test data reset (see "Testing with Persistent Data")

Directory Structure

my-extension/ ├── src/ │ └── ... ├── tests/ │ └── e2e/ │ ├── .auth/ │ │ └── user.json # Auth state (gitignored) │ ├── auth.setup.ts # Authentication │ └── my-extension.spec.ts ├── playwright.config.ts ├── .env # Gitignored ├── .env.example └── package.json

Patterns

Test Fixtures

import { test } from '@umbraco/playwright-testhelpers';

test('my test', async ({ umbracoApi, umbracoUi }) => { // umbracoApi - API helpers for setup/teardown // umbracoUi - UI helpers for backoffice interaction });

AAA Pattern (Arrange-Act-Assert)

test('can create content', async ({ umbracoApi, umbracoUi }) => { // Arrange - Setup via API await umbracoApi.documentType.createDefaultDocumentType('TestDocType');

// Act - Perform user actions via UI await umbracoUi.goToBackOffice(); await umbracoUi.content.goToSection(ConstantHelper.sections.content); await umbracoUi.content.clickActionsMenuAtRoot();

// Assert - Verify results expect(await umbracoApi.document.doesNameExist('TestContent')).toBeTruthy(); });

Idempotent Cleanup

test.afterEach(async ({ umbracoApi }) => { await umbracoApi.document.ensureNameNotExists(contentName); await umbracoApi.documentType.ensureNameNotExists(documentTypeName); });

API Helpers (umbracoApi)

Document Types:

await umbracoApi.documentType.createDefaultDocumentType('TestDocType'); await umbracoApi.documentType.createDocumentTypeWithPropertyEditor( 'TestDocType', 'Textstring', dataTypeData.id ); await umbracoApi.documentType.ensureNameNotExists('TestDocType');

Documents:

await umbracoApi.document.createDefaultDocument('TestContent', docTypeId); await umbracoApi.document.createDocumentWithTextContent( 'TestContent', docTypeId, 'value', 'Textstring' ); await umbracoApi.document.publish(contentId); await umbracoApi.document.ensureNameNotExists('TestContent');

Data Types:

const dataType = await umbracoApi.dataType.getByName('Textstring'); await umbracoApi.dataType.create('MyType', 'Umbraco.TextBox', 'Umb.PropertyEditorUi.TextBox', []);

Using Builders for Complex Data

For complex test data, use @umbraco/json-models-builders :

import { DocumentTypeBuilder, DocumentBuilder } from '@umbraco/json-models-builders';

test('create complex document type', async ({ umbracoApi }) => { // Build a document type with multiple properties const docType = new DocumentTypeBuilder() .withName('Article') .withAlias('article') .addGroup() .withName('Content') .addTextBoxProperty() .withAlias('title') .withLabel('Title') .done() .addRichTextProperty() .withAlias('body') .withLabel('Body') .done() .done() .build();

await umbracoApi.documentType.create(docType); });

Why use builders?

  • Fluent API makes complex structures readable

  • Ensures valid payload structure for Umbraco API

  • Handles required fields and defaults

  • Type-safe in TypeScript

UI Helpers (umbracoUi)

Navigation:

await umbracoUi.goToBackOffice(); await umbracoUi.content.goToSection(ConstantHelper.sections.content); await umbracoUi.content.goToContentWithName('My Page');

Testing Custom Trees in Sidebar

When testing custom tree extensions (e.g., in Settings), use this pattern to handle async loading and scrolling:

test('should click custom tree item', async ({ umbracoUi }) => { await umbracoUi.goToBackOffice(); await umbracoUi.content.goToSection(ConstantHelper.sections.settings);

// 1. Wait for your tree heading (custom trees often at bottom of sidebar) await umbracoUi.page.getByRole('heading', { name: 'My Tree' }).waitFor({ timeout: 15000 });

// 2. Scroll into view (important - sidebar may be long) await umbracoUi.page.getByRole('heading', { name: 'My Tree' }).scrollIntoViewIfNeeded();

// 3. Wait for tree items to load (async from API) const item1Link = umbracoUi.page.getByRole('link', { name: 'Item 1' }); await item1Link.waitFor({ timeout: 15000 });

// 4. Click the item await item1Link.click();

// Assert workspace loads await expect(umbracoUi.page.locator('my-tree-workspace-editor')).toBeVisible({ timeout: 15000 }); });

Why this pattern?

  • Custom trees are often at the bottom of the Settings sidebar

  • Tree items load asynchronously from your API

  • Using getByRole('link', { name: '...' }) is more reliable than generic umb-tree-item selectors

  • Built-in trees (Document Types, etc.) also use umb-tree-item , causing selector conflicts

Content Actions:

await umbracoUi.content.clickActionsMenuAtRoot(); await umbracoUi.content.clickCreateActionMenuOption(); await umbracoUi.content.chooseDocumentType('TestDocType'); await umbracoUi.content.enterContentName('My Page'); await umbracoUi.content.enterTextstring('My text value'); await umbracoUi.content.clickSaveButton();

Constants:

import { ConstantHelper } from '@umbraco/playwright-testhelpers';

ConstantHelper.sections.content ConstantHelper.sections.settings ConstantHelper.buttons.save ConstantHelper.buttons.saveAndPublish

Examples

Complete Test

import { expect } from '@playwright/test'; import { ConstantHelper, NotificationConstantHelper, test } from '@umbraco/playwright-testhelpers';

const contentName = 'TestContent'; const documentTypeName = 'TestDocType'; const dataTypeName = 'Textstring'; const contentText = 'Test content text';

test.afterEach(async ({ umbracoApi }) => { await umbracoApi.document.ensureNameNotExists(contentName); await umbracoApi.documentType.ensureNameNotExists(documentTypeName); });

test('can create content', { tag: '@smoke' }, async ({ umbracoApi, umbracoUi }) => { // Arrange const dataTypeData = await umbracoApi.dataType.getByName(dataTypeName); await umbracoApi.documentType.createDocumentTypeWithPropertyEditor( documentTypeName, dataTypeName, dataTypeData.id );

// Act await umbracoUi.goToBackOffice(); await umbracoUi.content.goToSection(ConstantHelper.sections.content); await umbracoUi.content.clickActionsMenuAtRoot(); await umbracoUi.content.clickCreateActionMenuOption(); await umbracoUi.content.chooseDocumentType(documentTypeName); await umbracoUi.content.enterContentName(contentName); await umbracoUi.content.enterTextstring(contentText); await umbracoUi.content.clickSaveButton();

// Assert await umbracoUi.content.waitForContentToBeCreated(); expect(await umbracoApi.document.doesNameExist(contentName)).toBeTruthy(); const contentData = await umbracoApi.document.getByName(contentName); expect(contentData.values[0].value).toBe(contentText); });

test('can publish content', async ({ umbracoApi, umbracoUi }) => { // Arrange const dataTypeData = await umbracoApi.dataType.getByName(dataTypeName); const docTypeId = await umbracoApi.documentType.createDocumentTypeWithPropertyEditor( documentTypeName, dataTypeName, dataTypeData.id ); await umbracoApi.document.createDocumentWithTextContent( contentName, docTypeId, contentText, dataTypeName );

// Act await umbracoUi.goToBackOffice(); await umbracoUi.content.goToSection(ConstantHelper.sections.content); await umbracoUi.content.clickActionsMenuForContent(contentName); await umbracoUi.content.clickPublishActionMenuOption(); await umbracoUi.content.clickConfirmToPublishButton();

// Assert await umbracoUi.content.doesSuccessNotificationHaveText( NotificationConstantHelper.success.published ); const contentData = await umbracoApi.document.getByName(contentName); expect(contentData.variants[0].state).toBe('Published'); });

Working Example: tree-example

The tree-example demonstrates E2E testing for a custom tree extension:

Location: umbraco-backoffice/examples/tree-example/Client/

Run E2E tests (requires running Umbraco)

URL=https://localhost:44325
UMBRACO_USER_LOGIN=admin@example.com
UMBRACO_USER_PASSWORD=yourpassword
npm run test:e2e # 7 tests

Key files:

  • tests/playwright.e2e.config.ts

  • E2E configuration with auth setup

  • tests/auth.setup.ts

  • Authentication using testhelpers

  • tests/tree-e2e.spec.ts

  • Tests for custom tree in Settings sidebar

Working Example: notes-wiki (Full-Stack with Data Reset)

The notes-wiki demonstrates E2E testing with persistent data and CRUD operations:

Location: umbraco-backoffice/examples/notes-wiki/Client/

Run E2E tests (with data reset)

URL=https://localhost:44325
UMBRACO_USER_LOGIN=admin@example.com
UMBRACO_USER_PASSWORD=yourpassword
UMBRACO_DATA_PATH=/path/to/App_Data
npm run test:e2e # 16 tests

Key files:

  • tests/playwright.e2e.config.ts

  • Config with globalSetup for data reset

  • tests/global-setup.ts

  • Resets data to seed state before tests

  • tests/test-seed-data.json

  • Known test data (notes, folders)

  • tests/notes-wiki-e2e.spec.ts

  • CRUD and navigation tests

What it demonstrates:

  • Testing a custom section using goToSection('notes')

  • Resetting file-based data before each test run

  • Testing tree navigation, folders, and workspaces

  • Entity actions via "View actions" button (more reliable than right-click)

  • Dashboard and workspace view testing

Testing Extensions with Persistent Data

When your extension persists data (JSON files, database, etc.), tests need predictable starting state.

Global Setup Pattern

Add globalSetup to reset data before tests:

playwright.e2e.config.ts:

export default defineConfig({ // ... other config globalSetup: './global-setup.ts', });

global-setup.ts:

import { FullConfig } from '@playwright/test'; import * as fs from 'fs'; import * as path from 'path';

async function globalSetup(config: FullConfig) { const dataPath = process.env.UMBRACO_DATA_PATH; if (!dataPath) { console.warn('⚠️ UMBRACO_DATA_PATH not set. Skipping data reset.'); return; }

const targetFile = path.join(dataPath, 'MyExtension/data.json'); const seedFile = path.join(__dirname, 'test-seed-data.json');

// Ensure directory exists fs.mkdirSync(path.dirname(targetFile), { recursive: true });

// Copy seed data to target fs.copyFileSync(seedFile, targetFile); console.log('🌱 Reset data to seed state'); }

export default globalSetup;

test-seed-data.json:

{ "items": [ { "id": "test-1", "name": "Test Item 1" }, { "id": "test-2", "name": "Test Item 2" } ] }

Environment Variable

Add UMBRACO_DATA_PATH to locate your Umbraco's App_Data folder:

UMBRACO_DATA_PATH=/path/to/Umbraco.Web.UI/App_Data npm run test:e2e

Testing Custom Sections

Custom sections work with testhelpers' goToSection() method - pass the section pathname:

// Section pathname - matches what you defined in section/constants.ts const MY_SECTION = 'my-section';

// Helper to navigate to custom section using testhelpers async function goToMySection(umbracoUi: any) { await umbracoUi.goToBackOffice(); await umbracoUi.content.goToSection(MY_SECTION); await umbracoUi.page.waitForTimeout(500); }

test('should navigate to custom section', async ({ umbracoUi }) => { await goToMySection(umbracoUi);

// Assert - your dashboard or tree should be visible await expect(umbracoUi.page.getByText('Welcome')).toBeVisible({ timeout: 15000 }); });

// To verify the section exists in the section bar: test('should display my section', async ({ umbracoUi }) => { await umbracoUi.goToBackOffice(); await expect(umbracoUi.page.getByRole('tab', { name: 'My Section' })).toBeVisible({ timeout: 15000 }); });

Context Menu (Entity Actions) Testing

Testing entity actions on tree items. Uses umbracoUi.page since testhelpers don't cover custom entity actions.

Important: Entity actions in Umbraco are rendered as buttons inside the dropdown menu, not as menuitem roles directly. The most reliable approach is to use the "View actions" button rather than right-click:

test('should show delete action via actions button', async ({ umbracoUi }) => { await goToMySection(umbracoUi);

// Wait for tree item const itemLink = umbracoUi.page.getByRole('link', { name: 'My Item' }); await itemLink.waitFor({ timeout: 15000 });

// Hover to reveal action buttons await itemLink.hover();

// Click the "View actions" button to open dropdown const actionsButton = umbracoUi.page.getByRole('button', { name: "View actions for 'My Item'" }); await actionsButton.click();

// Wait for dropdown and check for actions (actions are BUTTONS, not menuitems!) await umbracoUi.page.waitForTimeout(500); const deleteButton = umbracoUi.page.getByRole('button', { name: 'Delete' }); const renameButton = umbracoUi.page.getByRole('button', { name: 'Rename' });

// Assert - at least one action should be visible await expect(deleteButton.or(renameButton)).toBeVisible({ timeout: 5000 }); });

test('should delete item via actions menu', async ({ umbracoUi }) => { await goToMySection(umbracoUi);

const itemLink = umbracoUi.page.getByRole('link', { name: 'Item to Delete' }); await itemLink.waitFor({ timeout: 15000 });

// Hover and open actions menu await itemLink.hover(); await umbracoUi.page.getByRole('button', { name: "View actions for 'Item to Delete'" }).click();

// Click delete button await umbracoUi.page.getByRole('button', { name: 'Delete' }).click();

// Confirm deletion (if modal appears) const confirmButton = umbracoUi.page.getByRole('button', { name: /Confirm|Delete/i }); if (await confirmButton.isVisible({ timeout: 2000 }).catch(() => false)) { await confirmButton.click(); }

// Assert - item should be gone await expect(itemLink).not.toBeVisible({ timeout: 15000 }); });

Alternative: Right-Click Context Menu

Right-click also works but the actions button approach is more reliable:

// Right-click approach (less reliable than actions button) await itemLink.click({ button: 'right' }); await umbracoUi.page.waitForTimeout(500); await umbracoUi.page.getByRole('button', { name: 'Delete' }).click();

CRUD Testing Patterns

For custom extensions, use umbracoUi.page for UI interactions. For core Umbraco content, prefer umbracoApi helpers for setup/teardown.

Create via Actions Menu (Custom Extension)

test('should create new item', async ({ umbracoUi }) => { await goToMySection(umbracoUi);

// Hover over parent folder and use Create button const folderLink = umbracoUi.page.getByRole('link', { name: 'Parent Folder' }); await folderLink.hover();

// Scope to specific menu to avoid ambiguity with multiple items const folderMenu = umbracoUi.page.getByRole('menu').filter({ hasText: 'Parent Folder' }); const createButton = folderMenu.getByRole('button', { name: 'Create Note' }); await createButton.click();

// Assert - workspace should open with "New" indicator await expect(umbracoUi.page.locator('my-workspace-editor')).toBeVisible({ timeout: 15000 }); });

Scoping to Specific Tree Items

When multiple tree items exist with similar elements, scope selectors to avoid ambiguity:

// WRONG - ambiguous when multiple folders have "Create" buttons const createButton = page.getByRole('button', { name: 'Create' });

// CORRECT - scoped to specific folder's menu const folderMenu = page.getByRole('menu').filter({ hasText: 'My Folder' }); const createButton = folderMenu.getByRole('button', { name: 'Create' });

Update and Save

test('should update item', async ({ umbracoUi }) => { await goToMySection(umbracoUi);

// Navigate to item await umbracoUi.page.getByRole('link', { name: 'Test Item' }).click(); await umbracoUi.page.locator('my-workspace-editor').waitFor({ timeout: 15000 });

// Update field const titleInput = umbracoUi.page.locator('uui-input#title'); await titleInput.clear(); await titleInput.fill('Updated Title');

// Save await umbracoUi.page.getByRole('button', { name: /Save/i }).click();

// Wait for save to complete await umbracoUi.page.waitForTimeout(2000);

// Assert - header should reflect change await expect(umbracoUi.page.getByText('Updated Title')).toBeVisible(); });

Running Tests

Run all E2E tests

npm run test:e2e

Run with UI mode (visual debugging)

npm run test:e2e:ui

Run specific test file

npx playwright test tests/e2e/my-extension.spec.ts

Run with specific tag

npx playwright test --grep "@smoke"

Run in debug mode

npx playwright test --debug

Troubleshooting

getByTestId() not finding elements

Ensure testIdAttribute: 'data-mark' is set in playwright.config.ts.

Authentication fails

  • Check .env credentials are correct

  • Ensure Umbraco instance is running

  • Verify STORAGE_STAGE_PATH is set

Tests timeout

  • Increase timeouts in config

  • Ensure Umbraco is responsive

  • Check for JS errors in browser console

Tests fail in CI

  • Ensure Umbraco instance is accessible

  • Set environment variables in CI

  • Use npx playwright install chromium

Alternative: MSW Mode (No Backend Required)

For faster testing without a real Umbraco backend, use the mocked backoffice approach.

Invoke: skill: umbraco-mocked-backoffice

Aspect Real Backend (this skill) MSW Mode

Setup Running Umbraco instance Clone Umbraco-CMS, npm install

Auth Required Not required

Speed Slower Faster

Use case Integration/acceptance UI/component testing

Source Transparency

This detail page is rendered from real SKILL.md content. Trust labels are metadata-based hints, not a safety guarantee.

Open in GitHubOpen in ClawHub

Related Skills

Related by shared tags or category signals.

General

umbraco-backoffice

No summary provided by upstream source.

Repository SourceNeeds Review
61-umbraco
General

umbraco-dashboard

No summary provided by upstream source.

Repository SourceNeeds Review
56-umbraco
General

umbraco-quickstart

No summary provided by upstream source.

Repository SourceNeeds Review
54-umbraco
General

umbraco-extension-template

No summary provided by upstream source.

Repository SourceNeeds Review
54-umbraco