testing-codegen

Use this skill for testing, code generation, and preparing contributions. Covers snapshot testing with insta , code generation commands, and changeset creation.

Safety Notice

This listing is imported from skills.sh public index metadata. Review upstream SKILL.md and repository scripts before running.

Copy this and send it to your AI assistant to learn

Install skill "testing-codegen" with this command: npx skills add biomejs/biome/biomejs-biome-testing-codegen

Purpose

Use this skill for testing, code generation, and preparing contributions. Covers snapshot testing with insta , code generation commands, and changeset creation.

Prerequisites

  • Install required tools: just install-tools (installs cargo-insta )

  • Install pnpm: corepack enable and pnpm install in repo root

  • Understand which changes require code generation

Common Workflows

Run Tests

Run all tests

cargo test

Run tests for specific crate

cd crates/biome_js_analyze cargo test

Run specific test

cargo test quick_test

Show test output (for dbg! macros)

cargo test quick_test -- --show-output

Run tests with just (uses CI test runner)

just test

Test specific crate with just

just test-crate biome_cli

Quick Test for Rules

Fast iteration during development:

// In crates/biome_js_analyze/tests/quick_test.rs // Modify the quick_test function:

const SOURCE: &str = r#" const x = 1; var y = 2; "#;

let rule_filter = RuleFilter::Rule("nursery", "noVar");

Run:

just qt biome_js_analyze

Quick Test for Parser Development

IMPORTANT: Use this instead of building full Biome binary for syntax inspection - it's much faster!

For inspecting AST structure when implementing parsers or working with embedded languages:

// In crates/biome_html_parser/tests/quick_test.rs // Modify the quick_test function:

#[test] pub fn quick_test() { let code = r#"<button on:click={handleClick}>Click</button>"#;

let source_type = HtmlFileSource::svelte(); let options = HtmlParserOptions::from(&source_type); let root = parse_html(code, options); let syntax = root.syntax();

dbg!(&syntax, root.diagnostics(), root.has_errors()); }

Run:

just qt biome_html_parser

The dbg! output shows the full AST tree structure, helping you understand:

  • How directives/attributes are parsed (e.g., HtmlAttribute vs SvelteBindDirective )

  • Whether values use HtmlString (quotes) or HtmlTextExpression (curly braces)

  • Token ranges and offsets needed for proper snippet creation

  • Node hierarchy and parent-child relationships

Snapshot Testing with Insta

Run tests and generate snapshots:

cargo test

Review generated/changed snapshots:

Interactive review (recommended)

cargo insta review

Accept all changes

cargo insta accept

Reject all changes

cargo insta reject

Review for specific test

cargo insta review --test-runner nextest

Snapshot commands:

  • a

  • accept snapshot

  • r

  • reject snapshot

  • s

  • skip snapshot

  • q

  • quit

Test Lint Rules

Test specific rule by name

just test-lintrule noVar

Run from analyzer crate

cd crates/biome_js_analyze cargo test

Create Test Files

Single file tests - Place in tests/specs/{group}/{rule}/ under the appropriate *_analyze crate for the language:

tests/specs/nursery/noVar/ ├── invalid.js # Code that should generate diagnostics ├── valid.js # Code that should not generate diagnostics └── options.json # Optional: rule configuration

File and folder naming conventions (IMPORTANT):

  • Use valid or invalid in file names or parent folder names to indicate expected behaviour.

  • Files/folders with valid in the name (but not invalid ) are expected to produce no diagnostics.

  • Files/folders with invalid in the name are expected to produce diagnostics.

  • When testing cases inside a folder, prefix the name of folder using valid /invalid e.g. validResolutionReact /invalidResolutionReact

tests/specs/nursery/noShadow/ ├── invalid.js # should generate diagnostics ├── valid.js # should not generate diagnostics ├── validResolutionReact/ └───── file.js # should generate diagnostics └── file2.js # should not generate diagnostics

Multiple test cases - Use .jsonc files with arrays:

// tests/specs/nursery/noVar/invalid.jsonc [ "var x = 1;", "var y = 2; var z = 3;", "for (var i = 0; i < 10; i++) {}" ]

Test-specific options - Create options.json :

{ "linter": { "rules": { "nursery": { "noVar": { "level": "error", "options": { "someOption": "value" } } } } } }

Top-Level Comment Convention (REQUIRED)

Every test spec file must begin with a top-level comment declaring whether it expects diagnostics. The test runner (assert_diagnostics_expectation_comment in biome_test_utils ) enforces this and panics if the rules are violated.

Write the marker text using whatever comment syntax the language under test supports. For languages that do not support comments at all, rely on the file/folder naming convention (valid /invalid ) instead.

For files whose name contains "valid" (but not "invalid"):

The comment is mandatory — the test panics if it is absent.

For files whose name contains "invalid" (or other names):

The comment is strongly recommended and is also enforced when present: if the comment says should generate diagnostics but no diagnostics appear, the test panics.

Rules enforced by the test runner:

File name contains Comment present? Behaviour

"valid" (not "invalid") should not generate diagnostics

Passes if no diagnostics

"valid" (not "invalid") should generate diagnostics

Passes if diagnostics present

"valid" (not "invalid") absent PANIC — comment is mandatory

"invalid" or neutral name should not generate diagnostics

Passes if no diagnostics

"invalid" or neutral name should generate diagnostics

Passes if diagnostics present

"invalid" or neutral name absent No enforcement (but add it anyway)

Important details:

  • The comment is found by scanning the entire file's leading trivia — it does not have to be literally the first token, but putting it at the very top (line 1) is the established convention.

  • Fixture/support files (e.g. foo.js , bar.ts ) that don't contain "valid" or "invalid" in their name do not require a comment, since they are not considered "valid test files" by the runner.

  • Files excluded from comment enforcement regardless of name: .snap , .json , .jsonc , .svelte , .vue , .astro , .html .

Code Generation Commands

After modifying analyzers/lint rules (during development):

just gen-rules # Updates rule registrations in *_analyze crates just gen-configuration # Updates configuration schemas

These lightweight commands generate enough code to compile and test without errors.

Full analyzer codegen (optional — CI autofix handles this):

just gen-analyzer

This is a composite command that runs gen-rules , gen-configuration , gen-migrate , gen-bindings , lint-rules , and format . You typically don't need to run this locally — the CI autofix job does it automatically when you open a PR.

After modifying grammar (.ungram files):

Specific language

just gen-grammar html

Multiple languages

just gen-grammar html css

All languages

just gen-grammar

After modifying formatters:

just gen-formatter html

After modifying configuration:

just gen-bindings

Generates TypeScript types and JSON schema.

Full codegen (rarely needed):

just gen-all

Before committing:

just ready

Runs full codegen + format + lint (takes time).

Or run individually:

just f # Format Rust and TOML just l # Lint code

Create Changeset

For user-visible changes (bug fixes, new features):

just new-changeset

This prompts for:

  • Package selection: Usually @biomejs/biome

  • Change type:

  • patch

  • Bug fixes

  • minor

  • New features

  • major

  • Breaking changes (requires targeting next branch)

  • Description: What changed (used in CHANGELOG)

Changeset writing guidelines:

  • Be concise and clear (1-3 sentences)

  • Start bug fixes with: Fixed #issue: ...

  • Use past tense for your actions: "Added", "Fixed", "Changed"

  • Use present tense for Biome behavior: "Biome now supports..."

  • Include code examples for new rules/features

  • Link to rules: useConst

  • End sentences with periods

Example changeset:

"@biomejs/biome": patch

Fixed #1234: The rule noVar now correctly handles variables in for loops.

Biome now analyzes the scope of loop variables properly.

Edit changeset - Files created in .changeset/ directory, edit them directly.

Run Doctests

Test code examples in documentation comments:

just test-doc

Debugging Tests

Use dbg!() macro in Rust code:

fn some_function() -> &'static str { let some_variable = "debug_value"; dbg!(&some_variable); // Prints during test some_variable }

Run with output:

cargo test test_name -- --show-output

Tips

  • Snapshot organization: Group by feature/rule in separate directories

  • Test both valid and invalid: Create both valid.js and invalid.js files

  • Options per folder: options.json applies to all tests in that folder

  • .jsonc arrays: Use for multiple quick test cases in script context (no imports/exports)

  • Code generation order: Grammar → Analyzer → Formatter → Bindings

  • CI compatibility: Use just commands when possible (matches CI)

  • Changeset timing: Create before opening PR, can edit after

  • Snapshot review: Always review snapshots carefully - don't blindly accept

  • Test performance: Use #[ignore] for slow tests, run with cargo test -- --ignored

  • Parser inspection: Use just qt <package> to run quick_test and inspect AST, NOT full Biome builds (much faster)

For general Biome development tips (string extraction, borrow checker patterns, legacy syntax), see the biome-developer skill.

Common Test Patterns

// Snapshot test in rule file #[test] fn test_rule() { assert_lint_rule! { noVar, invalid => [ "var x = 1;", "var y = 2;", ], valid => [ "const x = 1;", "let y = 2;", ] } }

// Quick test pattern #[test] #[ignore] // Uncomment when using fn quick_test() { const SOURCE: &str = r#" var x = 1; "#;

let rule_filter = RuleFilter::Rule("nursery", "noVar"); // Test runs with this configuration }

Code Generation Dependencies

When you modify... Run during dev... Full (optional, CI does this)

.ungram grammar files just gen-grammar <lang>

—

Lint rules in *_analyze

just gen-rules && just gen-configuration

just gen-analyzer

Formatter in *_formatter

just gen-formatter <lang>

—

Configuration types just gen-bindings

—

Before committing just f && just l

—

Full rebuild — just gen-all (slow)

References

  • Main testing guide: CONTRIBUTING.md § Testing

  • Insta documentation: https://insta.rs

  • Analyzer testing: crates/biome_analyze/CONTRIBUTING.md § Testing

  • Changeset guide: CONTRIBUTING.md § Changelog

Source Transparency

This detail page is rendered from real SKILL.md content. Trust labels are metadata-based hints, not a safety guarantee.

Open in GitHubOpen in ClawHub

Related Skills

Related by shared tags or category signals.

Coding

biome-developer

No summary provided by upstream source.

Repository SourceNeeds Review
-56
biomejs
Coding

parser-development

No summary provided by upstream source.

Repository SourceNeeds Review
-55
biomejs
Coding

formatter-development

No summary provided by upstream source.

Repository SourceNeeds Review
-49
biomejs