AI Test Generation Widens Brittleness in DOM‑Centric End‑to‑End Automation, Authors Say

Amanul Chowdhury and Vinay Gummadavelli argue that AI‑driven test generation is exposing a core limit of modern end‑to‑end testing: AI scales whatever abstraction it is given, and when that abstraction is the DOM it scales structural brittleness rather than robustness. They label this dynamic an "AI productivity paradox" for test automation and trace emergent reliability gaps to architecture — the DOM‑centric model — not merely to tooling.

Mainstream automation frameworks such as Playwright and Cypress operate against the DOM rather than the rendered user experience. Features like Playwright’s auto‑wait are DOM‑stability heuristics that ensure a node exists, is marked visible and not detached, but those checks do not guarantee event listeners are bound, asynchronous application state has settled, or that the final rendered layout matches human perception. In server‑side rendering flows (examples: React, Next.js) these mismatches can produce hydration gaps and layout shifts that lead to so‑called "ghost interactions."

The industry shift toward AI‑driven test creation compounds the problem because autonomous agents can generate dozens or thousands of scenarios in minutes, lowering the bar to test creation while multiplying fragile assets. When agents derive tests from code structure instead of the visual interface, they tend to anchor assertions to brittle XPaths or ephemeral CSS classes, producing a hidden maintenance backlog rather than durable coverage. Agentic auto‑execution also lacks the natural hesitation a human tester gives to wait for layout or hydration to complete.

Those brittle tests have concrete consequences: an automated click can succeed against a DOM node yet miss the user‑facing intent if the element is not interactable or perceptible, and DOM heuristics alone will not detect perceptual mismatches. Chowdhury and Gummadavelli therefore argue that improving reliability requires simultaneous validation across three dimensions: structure (DOM), perception (visual/UX), and business intent.

To close the gap they propose a hybrid perceptual pipeline that combines three elements: browser instrumentation to capture DOM and timing signals, agentic vision models to interpret rendered output the way humans do, and explicit intent validation to confirm business outcomes. This architecture is intended to surface perceptual mismatches and ghost interactions that DOM heuristics miss and to align automated checks with real user experience.

For builders the practical implications are clear: avoid relying solely on DOM assertions; add perceptual or visual checks; monitor hydration and layout‑shift windows; verify that event listeners are bound and async state has settled before asserting results; and prefer selector strategies resilient to UI churn. Adopting vision models and explicit intent checks will require changes to tooling and pipelines, but the authors contend it is the most direct path to reducing breakage and the maintenance backlog created by mass automated test generation.