How to automate QA testing with AI: six steps to a suite that tests itself.
To automate QA testing with AI, you inventory the manual regression cycle and stabilise the selectors and test data the suite will depend on. You write the framework rules before generating anything, then generate coverage from the product and each code change. You run it on every build, let it heal the breakages that come from intentional changes, and finally gate the release and retire the manual pass.
This is the method behind the pipeline Paul Rahme (Aldenebai) built at a US marina-software company, which removed about 80% of manual QA effort and ended dedicated manual regression cycles, rounded from production data, 2025–2026.
From a manual regression cycle to a pipeline that tests itself
Steps two and three are the ones teams skip, and they are the ones that decide whether generated tests are an asset or a second backlog.
Inventory the regression cycle
List what is tested by hand before each release, who does it, how long it takes, and what reached production broken anyway. The total hours are the number to beat, and the escaped defects tell you which flows deserve coverage first. Without this list, "we automated QA" has nothing to be measured against.
Stabilise selectors and test data
Give the product stable test hooks so a test does not depend on a class name that changes with the design, and give the suite data it creates and destroys itself so runs do not collide. Generated tests inherit whatever instability already exists, which is why this comes before generation and not after the first flaky week.
Write the framework rules before generating anything
Page objects, so a UI change is fixed in one place. Signal-based waits, never fixed sleeps. Tags, so a pull request runs the ten tests that matter while the nightly run takes everything. One reporting format. A generated test that breaks a rule is rejected exactly like a human's would be. The scripted framework as a service →
Generate coverage from the product and the diff
Point the pipeline at the requirements, the product, and each code change, and have it propose tests for the flows that actually changed. Review the first batches closely: you are teaching it the house style, and once the shape is right the review becomes a skim.
Run on every build and on a schedule, then let it heal
The tagged subset runs on every pull request, the full suite runs nightly. The pipeline may repair tests broken by intentional change, such as a renamed selector or a moved label, and may never repair an assertion about behaviour. Every repair is shown to a person as a diff. Definition →
Gate the release and retire the manual pass
A red critical-path test blocks the release. Then delete the manual checklist item by item as coverage replaces it, rather than in one go. Keep exploratory testing: finding the thing nobody specified is judgement, and it is what the QA team should be spending the returned hours on.
What the numbers were, and how they were counted
At the marina-software client, on its the client product, the pipeline removed about 80% of manual QA effort and left no dedicated manual regression cycle per release, rounded from production data, 2025–2026. "Manual QA effort" means the hours a person spent running regression before a release, and the measurement note sits on the case study.
Before any AI was involved, the same discipline applied by hand cut manual regression by about 70% at Eurisko, on Shasha Media's web, TV, and mobile products, with end-to-end frameworks in Cypress, WebdriverIO, and TypeScript. That figure is rounded from project data at Eurisko, and it is the honest baseline: most of the gain is framework discipline, and the AI layer is what stops the suite decaying.
When scripted suites are still the right answer
When the product changes slowly, a hand-written suite is cheaper to own and easier to reason about, and generation adds a moving part for no return. When there is no framework yet, generation makes the mess larger, so the first engagement is the framework. When the flows are few but the consequences are severe, as in a regulated banking product, the value is in careful assertions written by a person who understands the risk, not in volume of coverage.
The comparison is written out in full, including where each one fails, in AI test automation vs scripted suites.
Questions about automating QA with AI
How long does it take to automate QA testing with AI?
The framework and the first tagged suites take weeks rather than quarters; retiring the manual pass takes as long as it takes coverage to reach the flows the manual pass protected. The proposal names the dates before anything starts, and what moves them is usually selector stability, not the AI.
Will generated tests just pass no matter what?
They will if nobody checks, which is why every generated test is reviewed like human code and rejected when it cannot fail. A test that passes unconditionally is worse than no test, because it buys false confidence. The pipeline is also barred from healing an assertion about behaviour, only the mechanics of finding an element.
Do we still need QA engineers?
Yes, for the part that is judgement: what quality means for this product, which flows matter, whether a real failure blocks the release, and where users will break things nobody specified. What disappears is writing the next hundred tests and repairing the fifty that broke because a button moved.
Which frameworks does this work with?
Playwright, Cypress, and WebdriverIO for web, Appium for native mobile, and real devices through BrowserStack, running in GitHub Actions or the CI you already have. The pipeline extends the framework you own rather than replacing it, so existing tests keep running from day one.
Related pages
Step one, done with you, in 30 minutes.
The free Automation Audit inventories your regression cycle and writes down what a pipeline would remove first. Yours to keep either way.