Token-Efficient E2E Testing with Claude Code & Playwright

How to cut AI-assisted testing costs by 10× while getting better coverage — using a three-tier strategy that keeps screenshots out of your context window.

1 The Problem with Interactive AI Testing

When Claude Code drives a Playwright browser interactively — click, snapshot, click, snapshot — each step costs tokens. A full user journey of 15 interactions easily burns 30,000–60,000 tokens in a single session, mostly on accessibility snapshots that look like this:

### Snapshot
- Heading "Fleet Dashboard" [level=1]
- Button "+ Bootstrap Node" [ref=e47]
- Table [ref=e48]
  - Row [ref=e49]
    - Cell "mac-mini-01" [ref=e50]
    - Cell "online" [ref=e51]
    ...

Multiply that across 20 journeys and you’ve spent a meaningful chunk of a context window just describing what a browser sees. Then the context fills, compacts, and the next session starts cold.

The real cost: it’s not just API tokens. It’s session continuity. When context overflows mid-journey, you lose test state, have to re-login, re-navigate, and re-discover what was already verified.

2 The Three-Tier Testing Pyramid

The solution is to match the test tool to the question being answered. Not every test needs a browser. Not every assertion needs a screenshot.

The wider the tier, the more tests live there. The narrower the tier, the fewer tokens it must burn.

Tier 1 · API

~200

tokens / test

Tier 2 · Spec file

~2k

tokens / full run

Tier 3 · Interactive

~50k

tokens / journey

The numbers make the decision obvious: default to Tier 1, only escalate to Tier 2 for UI flows, and use Tier 3 only to debug specific failures.

3 Tier 1 — API Tests with pytest + httpx

Pure API tests have zero browser overhead. They validate backend logic, auth, validation, and business rules — which is the majority of what matters. Claude reads a compact pass/fail output, not screenshots.

What belongs here

Authentication flows, CRUD endpoints, pagination, status transitions, rate limiting, permission enforcement, edge-case inputs, and all security checks (path traversal, SQL injection probes).

# tests/integration/test_bootstrap.py
async def test_bootstrap_409_if_already_running(client, db):
    # First bootstrap — should succeed
    r1 = await client.post("/api/v1/ansible/bootstrap",
        json={"minion_id": "mac-01", "target_ip": "203.0.113.11"})
    assert r1.status_code == 200

    # Second bootstrap on same node — must be 409
    r2 = await client.post("/api/v1/ansible/bootstrap",
        json={"minion_id": "mac-01", "target_ip": "203.0.113.11"})
    assert r2.status_code == 409

Running 50 of these costs roughly the same as one interactive Playwright journey. The output Claude reads back is just:

50 passed in 4.32s

Rule: If the assertion doesn’t require seeing pixels, it belongs in Tier 1.

4 Tier 2 — Playwright Spec Files (Not Interactive)

The key insight: writing and running Playwright spec files is fundamentally different from interactive Playwright MCP. When you run npx playwright test --reporter=line, the only thing that comes back into Claude’s context is the text output — not snapshots, not screenshots.

The spec file is written once by Claude in your session. Subsequent runs — from CI, from the kri.sh test command, from your terminal — consume near-zero AI tokens. Only failures come back into context for diagnosis.

Spec file structure for kri

tests/e2e/
  helpers.ts ← login helpers, API base URL
  auth.spec.ts ← AUTH-01..07 (API + UI)
  fleet.spec.ts ← FLEET-01..14 (API + UI)
  bootstrap.spec.ts← BOOT-01..22 (API + UI)
  nodes.spec.ts ← NODE-01..18 (API + UI)
  baselines.spec.ts← BASE-01..12 (API + UI)
  groups.spec.ts ← GRP-01..12 (API + UI)
  playbooks.spec.ts← PLAY-01..21 (API + UI)
playwright.config.ts ← screenshots only-on-failure

The login trick that saves the most tokens

The loginViaApi helper bypasses the login UI entirely — it calls the auth endpoint with request.post() and injects tokens directly into localStorage. This skips 4 interactive steps (navigate, fill email, fill password, click submit) on every single test.

// helpers.ts — fast login via API, skips UI
export async function loginViaApi(page: Page, user = ADMIN) {
  const res = await page.request.post(`${API}/auth/login`, {
    data: { email: user.email, password: user.password },
  })
  const { access_token, refresh_token } = await res.json()
  await page.goto('/')
  await page.evaluate(({ at, rt }) => {
    localStorage.setItem('access_token', at)
    localStorage.setItem('refresh_token', rt)
  }, { at: access_token, rt: refresh_token })
  await page.goto('/fleet')
}

5 Tier 3 — Interactive MCP for Failure Diagnosis

Interactive Playwright MCP (navigating, clicking, snapshotting step-by-step inside a Claude session) is expensive but irreplaceable for one thing: diagnosing visual failures that text output can’t describe.

The subagent multiplier

When you do need interactive testing, spawn a subagent for it. The subagent’s context is isolated — all the snapshots it takes burn its token budget, not yours. It comes back with a one-paragraph summary.

# In your main session:
Agent({
  description: "Bootstrap modal interactive test",
  prompt: """Test the Bootstrap Node flow at http://localhost:5173:
    1. Login as admin@fleet.example.internal / changeme
    2. Click + Bootstrap Node
    3. Type mac-mini-01 — verify IP pre-fills and is locked
    4. Type brand-new-xyz — verify IP is editable
    5. Report PASS/FAIL for each step with any error text."""
})

The subagent does all the snapshotting internally. You receive one compact result. Your session context stays clean for the next task.

6 Running the Suite

All tests are wired into kri.sh. Make sure kri is running first.

# Start kri
./scripts/kri.sh start

# Run full E2E suite — gets compact line reporter output
./scripts/kri.sh test

# Run a specific file
./scripts/kri.sh test auth

# Run from repo root directly
npx playwright test --reporter=line

# Run only API-level tests (fastest)
npx playwright test --grep "API" --reporter=line

# Run with full HTML report (human review)
npx playwright test --reporter=html

What Claude reads back when all passes:

  Running 47 tests using 1 worker
  ··············································
  47 passed (38.2s)

What Claude reads back on failure:

  Running 47 tests using 1 worker
  ·······················✗····················
  1 failed

  ✗ bootstrap.spec.ts:89 › BOOT-03 existing minion ID auto-fills IP
    Error: Timed out 5000ms waiting for expect(locator).toBeVisible()
    Received: <p> "New node — enter IP address below" </p> visible

That failure message is 200 tokens. Claude knows exactly what to check without a screenshot.

7 Decision Reference

Question to answer	Tool	Cost
Does this API return the right status code?	pytest / Playwright API request	~200 tk
Does this UI flow work end-to-end?	Playwright spec file (headless)	~2k tk
Why is this specific test failing visually?	Subagent with interactive MCP	~10k tk
Full visual exploration of an unknown issue	Interactive MCP (direct, last resort)	~50k tk

The 10× savings come from two habits: writing spec files instead of interactive journeys, and spawning subagents for the rare times you need a real browser session. Both keep your main context window clean and your token bill predictable.

---