E2E Test Split Strategy

Splitting Playwright tests into CI-safe and @interactive categories for Tauri apps

E2E Test Split Strategy

Tauri apps face a unique testing challenge: the WebView runs on the OS's WebKit engine, and the app's keyboard shortcut engine does not reliably receive keyboard events in CI's headless browser. This article documents a practical split strategy that runs most tests in CI while reserving keyboard-dependent tests for local execution.

The Problem

Consider a test that opens the settings dialog:

test("opens settings dialog", async ({ page }) => {
  await page.keyboard.press("Control+,");
  await expect(page.getByTestId("settings-dialog")).toBeVisible();
});

This test works perfectly on a developer's machine but fails intermittently (or always) in CI because:

1. Headless browsers handle keyboard shortcuts differently -- The shortcut engine listens for keydown events, but headless Chromium may not dispatch them identically to a real browser

2. The app's shortcut engine needs focus -- In CI, the page may not have the right focus context for shortcuts to fire

3. Timing issues -- The shortcut engine initializes asynchronously; the test may send the key event before the engine is ready

Meanwhile, tests that use DOM APIs directly (clicking buttons, reading text content, asserting visibility) work reliably in both local and CI environments.

Two Test Categories

CI-Safe Tests

These tests interact with the DOM directly -- clicks, page.evaluate(), assertions on element visibility/content. They do not depend on keyboard shortcuts.

test.describe("Core Navigation", () => {
  test("navigates to archives via sidebar click", async ({ page }) => {
    await page.click('[data-testid="nav-archives"]');
    await expect(page.getByTestId("archives-page")).toBeVisible();
  });
});

@interactive Tests

These tests require the app's shortcut engine to process keyboard events. They are tagged with @interactive in the test.describe name:

test.describe("Settings Dialog @interactive", () => {
  test("opens with Ctrl+,", async ({ page }) => {
    await page.keyboard.press("Control+,");
    await expect(page.getByTestId("settings-dialog")).toBeVisible();
  });

  test("opens command palette with Ctrl+K", async ({ page }) => {
    await page.keyboard.press("Control+k");
    await expect(page.getByTestId("command-palette")).toBeVisible();
  });
});

CI Configuration

CI runs tests with --grep-invert to exclude @interactive tests:

# What CI runs (excludes @interactive)
pnpm test:e2e --project=chromium --grep-invert="@interactive"

Local development runs all tests:

# Run all tests locally (includes @interactive)
pnpm test:e2e

# Run only @interactive tests
pnpm test:e2e --grep="@interactive"

Test File Organization

Tests are organized by feature, not by category. Each spec file may contain both CI-safe and @interactive test blocks:

e2e/
  command-palette.spec.ts         # @interactive (keyboard shortcuts)
  core-navigation.spec.ts         # CI-safe (clicks)
  display-scale.spec.ts           # CI-safe (DOM assertions)
  draft-shortcut-guard.spec.ts    # @interactive (keyboard shortcuts)
  indent-guides.spec.ts           # CI-safe (visual assertions)
  list-indent-wrap.spec.ts        # CI-safe (editor behavior)
  panel-divider-drag.spec.ts      # CI-safe (mouse drag)
  settings-dialog.spec.ts         # @interactive (Ctrl+, to open)
  split-pane.spec.ts              # CI-safe (DOM assertions)
  write-and-archive.spec.ts       # CI-safe (clicks and text input)
  zoom-fix-verification.spec.ts   # CI-safe (CSS assertions)
  helpers.ts                      # Shared test utilities

WebKit Requirement

Since Tauri apps use the OS WebKit engine (not Chromium), all Playwright tests should use the WebKit browser for local verification:

// playwright.config.ts
export default defineConfig({
  projects: [
    {
      name: "webkit",
      use: { ...devices["Desktop Safari"] },
    },
  ],
});

CI uses Chromium for practical reasons (it is faster and more stable in headless mode), but the CI tests are explicitly designed to avoid WebKit-specific behaviors. The @interactive tests that exercise deeper browser integration are only run locally with WebKit.

Add a Native WebDriver Lane for Real Window Behavior

Playwright WebKit still drives a browser process, not the packaged Tauri application. It cannot prove behavior that depends on Tauri IPC, multiple native windows, AppKit focus, or the operating system's close lifecycle. Those cases do not have to remain entirely manual: on macOS, a dedicated test bundle can register tauri-plugin-wdio-webdriver and let WebdriverIO drive the real Tauri WebViews.

Keep this as a separate, serialized lane:

• Put the driver dependency behind an opt-in Cargo feature such as native-e2e; do not register it in normal builds.

• Use a dedicated Tauri config, bundle identifier, application name, state directory, and incognito WebView store so tests cannot touch a daily-driver installation or signed-in account.

• Grant only the embedded driver's capability to the test windows. Keep the production capability set free of driver permissions.

• Use W3C pointer and keyboard actions for interaction. JavaScript-dispatched DOM events can bypass the editor, focus, clipboard, and native-window paths the test is meant to prove.

• Set maxInstances: 1 when the embedded driver owns a fixed port or the journey creates real application windows.

• On failure, capture every discoverable window's screenshot plus sanitized title, URL, realm identifier, frontend error marker, and protocol trace. Multi-window bugs are otherwise hard to reconstruct from one selected WebDriver context.

• Always clean the dedicated fixture state, wait for the driver port to close, and terminate only the exact test-bundle executable if the runner is interrupted.

# src-tauri/Cargo.toml
[features]
native-e2e = ["dep:tauri-plugin-wdio-webdriver"]

[target.'cfg(target_os = "macos")'.dependencies]
tauri-plugin-wdio-webdriver = { version = "1.3.0", optional = true }

#[cfg(all(target_os = "macos", feature = "native-e2e"))]
let builder = builder.plugin(tauri_plugin_wdio_webdriver::init());

// wdio.conf.ts (excerpt)
export const config = {
  maxInstances: 1,
  hostname: "127.0.0.1",
  port: 45678,
  services: [["@wdio/tauri-service", { driverProvider: "embedded" }]],
  capabilities: [
    {
      browserName: "tauri",
      "tauri:options": { application: TEST_APP_BINARY },
    },
  ],
};

Add an explicit release-exclusion check as part of the lane: inspect the normal Cargo dependency graph without native-e2e, build the ordinary production bundle, and scan its capabilities and emitted assets for driver/plugin/test-bootstrap markers. A green native test is not enough if it accidentally ships its automation surface.

Writing New Tests

Follow this decision tree:

1. Does the test use keyboard shortcuts (Ctrl+K, Ctrl+,, etc.)?

• Yes -> Tag with @interactive

• No -> Continue

2. Does the test rely on focus-dependent behavior (shortcut engine, vim mode)?

• Yes -> Tag with @interactive

• No -> It is CI-safe, no tag needed

3. Shared utilities go in helpers.ts -- Page object patterns, common setup, and wait helpers.

Example of a well-structured test file with both categories:

import { test, expect } from "@playwright/test";

// CI-safe: uses DOM clicks
test.describe("Settings Display", () => {
  test("shows display scale options", async ({ page }) => {
    // Open settings via DOM click (not keyboard shortcut)
    await page.click('[data-testid="settings-button"]');
    await expect(page.getByTestId("display-scale-options")).toBeVisible();
  });
});

// Interactive: uses keyboard shortcut
test.describe("Settings Shortcut @interactive", () => {
  test("Ctrl+, opens settings", async ({ page }) => {
    await page.keyboard.press("Control+,");
    await expect(page.getByTestId("settings-dialog")).toBeVisible();
  });
});

Key Takeaway

The split strategy is pragmatic: run what you can in CI, run what you cannot locally. The @interactive tag is a simple grep-based filter that requires no special Playwright plugins or configuration. It ensures CI stays green while still allowing thorough local testing of keyboard-driven features.