Point it at a Playwright test failing on a changed selector. It runs the test, reads the failure, checks the real DOM, proposes a fix, and re-verifies it — capped at three attempts, and built to decline rather than mask a real product bug it isn't confident about.
A test fails because a data-testid got renamed, not because the feature broke. An SDET
re-diagnoses the same shape of failure over and over. That's a narrow, well-bounded job — which
also makes it exactly the wrong place to let a language model act on instinct alone.
This project treats the model as the reasoning engine and the diagnosis as untrusted input: two deterministic gates decide what the agent is allowed to touch, independent of anything the model itself concludes.
Playwright's actual failure text is pattern-matched into one of six categories before the agent is ever invoked. Locator timeouts and zero-match assertions are eligible. Value/content mismatches, navigation errors, and anything unrecognized are flagged straight to a human — the model never runs.
Every edit is checked regardless of what the model says about its own diagnosis. Only a single-line
Playwright locator/getBy* swap is accepted — anything touching expect(...),
spanning multiple lines, or changing text outside the locator call is rejected, with the reason handed
back so the model can retry narrower.
Against a demo test with a deliberately stale CSS selector, where the real element uses
data-testid="submit-btn" instead.
Gate 1: locator_timeout — eligible (Action or structural assertion timed out waiting for the locator — consistent with a stale selector.) → read_file("tests/fixtures/example.spec.ts") ✓ import { test, expect } from '../support/fixtures'; ... → inspect_dom("tests/fixtures/example.spec.ts") ✓ {"available":true,"snapshot":"- button \"Submit\""} → edit_file("tests/fixtures/example.spec.ts") - await page.locator('#submit-btn').click(); + await page.getByTestId('submit-btn').click(); ✓ {"path":"tests/fixtures/example.spec.ts"} → run_test("tests/fixtures/example.spec.ts") ✓ {"passed":true} run_test 1/3: PASSED Result: fixed after 1 run_test attempt(s).
inspect_dom only works for tests using the project's shared fixture wrapper, not arbitrary Playwright suites out of the box.