AI test automation vs scripted suites: what actually changes.
A scripted suite is end-to-end coverage written and maintained by engineers in Playwright, Cypress, WebdriverIO, or Selenium. AI test automation is a pipeline that reads the product and its changes, generates the coverage, runs it on a schedule, and repairs tests that broke because the product changed on purpose. Paul Rahme spent years building the first kind, at Eurisko, Saudi Tourism Authority, and a bank, before building the second at the marina-software client. This is what actually differs.
The comparison
| DIMENSION | SCRIPTED SUITE | AI TEST AUTOMATION |
|---|---|---|
| Who writes the tests | Engineers, by hand, from requirements and exploration. | The pipeline, from the product, the requirements, and the code changes; engineers review. |
| Who maintains them | Engineers, every time the UI or API changes. | The pipeline heals tests broken by intentional change; real regressions are reported. |
| Coverage over time | Grows as fast as people have time; usually behind the product. | Grows with the product, because generation runs on every change. |
| Flakiness | The chronic cost: fixed waits, shared state, order dependence. | Same root causes, but the pipeline can rewrite a flaky test against the framework's rules, then a person confirms. |
| Result at Eurisko / marina-software client | About 70% less manual regression with hand-built frameworks. | About 80% less manual QA effort, zero dedicated manual regression cycles per release. |
| Judgment | Stays with people. | Stays with people: what quality means, which flows matter, what to do with a real failure. |
| Risk | Coverage gaps nobody had time to fill. | A healed test that should have failed; mitigated by rules on what may be healed and a human review of every heal. |
| Prerequisite | A framework and a team. | A framework, CI, and a product that is at least partly documented. |
Rounded from production data; definitions on the self-testing QA case study.
What the scripted years taught
Frameworks fail for the same reasons everywhere: fixed timeouts instead of signals, tests that share state, suites that take forty minutes and are ignored when red. The discipline that fixes this, page objects, signal-based waits, isolated data, tags that drive selective runs, is what took manual regression down by about 70% at Eurisko before any model was involved.
The AI pipeline does not replace that discipline. It runs inside it. A generated test that ignores the framework's rules is rejected the same way a human's would be.
What the AI pipeline actually removes
Two jobs: writing the next hundred tests, and fixing the fifty that broke because a button moved. Those were most of a QA engineer's week. What remains is the judgment: which flows matter, whether a real failure blocks the release, where users will break the product in ways no suite anticipated.
The one new risk is a test that heals when it should have failed. The pipeline is allowed to heal only specific kinds of breakage, such as a selector or a label, never an assertion about behaviour, and every heal is shown to a person. How the pipeline works →
Questions QA and engineering leads ask
Do we throw away our existing Playwright or Cypress suite?
No. The pipeline extends the framework you have. Existing tests keep running; generation fills the gaps and takes over maintenance. If the framework has no rules yet, those come first.
How do we know a healed test is still testing the right thing?
By limiting what may be healed and reviewing every heal. Selectors and labels can be repaired automatically; assertions about behaviour cannot. A heal shows up as a diff a person approves, like any other change.
Does this replace the QA team?
It replaces the manual regression cycle and most test maintenance. Judgment about quality and risk stays with people, who now spend their time there. See the production case.
What does the pipeline need from us to start?
A codebase with CI, an existing or agreed framework, and enough documentation or requirements for the generator to read. The Automation Audit checks all three and tells you what to fix first.
Related pages
Find out what your release cycle would look like.
The free Automation Audit maps your release cycle and coverage gaps into a concrete pipeline plan. 30 minutes, written Automation Map included.