E2E tests in CI/CD with Playwright and Cypress

Why e2e testing falls apart on unstable environments

An e2e test run fails intermittently: the same checkout scenario is green today, red tomorrow at the confirmation step.

The cause is sought in build logs, but it is in the environment — a parallel run took the same test account, a migration did not roll back the schema, a previous scenario left a record in the cart.

Locally the same test passes ten times in a row, so the failure cannot be reproduced. On Playwright it looks the same: there is a trace, but no culprit.

False run failures cost more than it seems. Feedback time per branch grows from minutes to hours, and the share of flaky checks in the report reaches one third — such a metric cannot be used to judge release readiness.

Developers rerun failed checks without reading them, and a real regression drowns in noise. Then manual rechecking before release kicks in: critical paths are tested by hand, release is delayed, and confidence does not increase.

The root is that the environment is shared and data is not isolated. Until test accounts, database state, and run order are separated into independent environments, any edits to automated tests only mask the problem.

Test environment data isolation is not an optimization but a condition without which automated checks give no signal at all. Let's break down how this is solved.

What changes after embedding runs into the build pipeline

Key indicators showing that runs have become embedded in the pipeline rather than living as a separate nightly job. Each item is checked against logs and metrics, not the team's feelings.

• Build time. The full set of browser scenarios is split into parallel streams by specification files, browser warm-up and data preparation are moved out of the critical path — the run fits into minutes instead of stretching for hours.
• Share of false failures. Waiting for network responses and element appearance replaces fixed pauses, test data is isolated per run — the suite stops flickering on an unstable environment, and a red report means a real defect.
• Early defect detection. Checks run on every merge request: a break in the chain "form — request — service response — record in storage" is seen by the author of the change while the context is still fresh.
• Environment recovery time. The environment is raised from an image and seed in minutes; after a failure the run starts from a clean state instead of being fixed manually by the on-call engineer.
• Result observability. The report saves video, trace, and state snapshot at the failure step — investigation takes minutes instead of a "it doesn't reproduce for me" thread.
• Eliminated class of failures. Integration errors between services and regressions in user scenarios are caught before deployment, not in a customer request.

Playwright and Cypress address these items differently: the first is more convenient for parallel runs and multi-tab scenarios, the second for teams with already established work on it. We choose based on the nature of the application and which metrics need to move.

E2E testing run formats: regression, smoke, nightly

Run modes differ not in the set of tests but in when they are launched and what feedback they give. The smoke suite answers one question — is the application alive after deployment.

It runs on every build to the main branch, fits into minutes, and stops the pipeline before a broken version reaches the environment. The price for speed is narrow coverage: such a suite will not catch regression in deep scenarios.

Running the regression suite on every commit is pointless: it gives an answer after tens of minutes, and parallel execution by streams cuts this time proportionally to the number of build machines.

The nightly run takes on what is expensive to keep in the fast loop — the full suite, long scenarios, checks on several environments.

Parallel execution is not a separate type of checks but a way to fit regression into the workday: both tools support splitting the suite into parts, but require test data isolation and a separate environment per stream.

The working scheme is smoke on every commit, regression on the release branch, nightly for full coverage; the choice of a specific configuration depends on how long the team is willing to wait for a signal.

Implementation stages: from environment audit to handover of regulations

We carry out the work in stages, and each ends with an artifact that can be verified.

The order is not accidental: first we record the actual state of environments and scenarios, then we build the environment scheme, and only after that we configure runs — otherwise tests fail on infrastructure rather than code, and trust in the test suite is lost within a couple of weeks.

1. Audit of environments and scenarios. We inventory contours, accesses, and test data, collect a registry of critical user paths, and cut what is cheaper to check at a lower level. The output is a coverage map with explicit gaps.
2. Environment scheme. We separate contours by purpose: where the full suite runs, where the short smoke run runs, how data is prepared and state is rolled back between runs. This removes the main cause of false failures.
3. Run setup. We embed the Playwright and Cypress suite into the pipeline: on changes — a shortened list by affected areas, at night — the full one. The report shows not "twelve tests failed" but the specific path and step.
4. Stabilization of failures. We analyze every flaky test: explicit waits instead of fixed pauses, data isolation, rerun only on confirmed infrastructure failure. The target is a successful run rate above 95%.
5. Validation on real releases. We run the suite in parallel with several releases and compare signals with actual defects: it is clear whether it catches regressions or just makes noise.
6. Handover of regulations. We hand over documentation: how to add a test, how to read the report, who does what when it fails. After that the team maintains the suite itself.

Output artifacts: configurations, environment schemes, regulations

What remains with the customer is not a report on work but a set of artifacts in your repository: the e2e testing run comes up on a clean machine according to the instructions, without contacting us.

• Environment configurations — parameterized profiles of the environment and pre-production environment: addresses, timeouts, sets of test accounts. Secrets are stored in a secret store, and environment switching is done with a single run variable.
• Environment scheme — a connection diagram: where browser runs go, how test data is populated, what is needed to restore the environment from scratch.
• Data access rules — which accounts are used, which data is synthetic, how scenarios are isolated so that parallel runs do not overwrite each other's state.
• Run documentation — how to read the report, where traces and screenshots of failed steps are, how long artifacts are stored, how to rerun a single scenario locally.
• Release regulation — which suites block deployment, which are informational, who resolves a red run and within what timeframe.
• Procedure for investigating failures — the criterion for a flaky scenario and quarantine with a mandatory ticket for fixing, so that noise does not mask real defects.
• Helper modules — shared selectors, steps, and fixtures so that your engineers can add scenarios in Playwright or Cypress without us.

Acceptance is simple: an engineer who did not participate in the implementation runs the scenario according to the instructions from the repository. If it works, the set is complete and continues to work without us.

Case: release verification time reduced fourfold

Baseline

Regression before release was run manually according to a checklist: two engineers spent two days on it, and deployment waited for the verifiers, not the other way around.

About a third of red runs turned out to be false — the test did not wait for the interface to be ready, and some checks shared the same data and failed because of traces from a neighboring scenario.

Engineers manually rechecked such failures, and trust in automation fell so much that red reports stopped being read until the release itself.

What was changed

Each scenario received its own data set and a clean contour that is raised before the run and torn down after — there are no more intersections between checks. Waits were replaced with explicit element readiness conditions rather than fixed pauses.

Checks were moved to Playwright: the run goes in parallel in several streams, a failed scenario is rerun once while saving video and trace. Tests that still remained flaky are marked and do not block deployment, but go into a separate list for investigation.

Measurements after

The key here is not speed itself, but that verification stopped being a bottleneck: a release can be deployed on the day it is ready, not on the day the verifiers become available.

False failures stopped masking real ones — now every red report means a specific defect visible in the trace, not a reason to rerun and hope.

What to do if checks are already deployed but not trusted

The contour is deployed, but the run is red: checks are rerun until green, failures in the report are not read, and manual verification remains the main one before release. Trust in checks is broken not by tool choice but by the absence of metrics and an owner for the contour.

We start with numbers: the share of flaky runs per scenario, full run time, the share of failures that reached a real defect. Then sorting — some scenarios go to quarantine and repair, some close gaps in waits and test data isolation, duplicates are removed.

Migration to another tool goes by domain, not as a whole: new scenarios are written only in the new contour, old ones are migrated when the functionality is touched. The exit criterion is ten green runs in a row without reruns and a failure that is read as a task, not as noise.

Frequently asked questions about environments, data, and false failures

Doubts that arise before the start are usually not about the tool itself, but about what it will cost to maintain. Below are answers to what engineers and department managers most often ask before putting an e2e run into the common build pipeline.

Is a separate environment needed or is a shared test contour enough?

A separate one is needed: on a shared contour, automated runs and manual regression interfere with each other — checks change data and break each other's state.

The environment is raised from the same infrastructure descriptions as production, but at a smaller scale, with stubs for external systems.

Where should test data come from?

We prepare it before the scenario through the application's programmatic interfaces, with unique prefixes, and remove it after the run. Then runs do not conflict with each other, and the database remains in a predictable state.

Will scenarios have to be rewritten after every release?

We describe scenarios at the level of user actions and business meaning, not page structure, so redesign does not break them.

Maintenance comes down to adding new paths and removing outdated ones: we keep manual regression for exploratory and visual checks, while automation holds critical routes.

Let's discuss the task: what to prepare for the start of work

The start of work is accelerated not by kilobytes of documentation but by three things: an environment where checks can run, a list of scenarios that must not break, and people who make decisions.

Without them, the team spends the first week in back-and-forth instead of writing tests. Below is the minimum that removes most questions at the input.

• Access to the repository and build pipeline: at the start, read permissions and the ability to run the build manually are enough.
• A test environment with an isolated database, predictable accounts and roles, and the ability to return data to its initial state between runs.
• A list of critical scenarios — 10–20 paths whose failure is most expensive: login, payment, checkout, report export.
• A decision on external services: where we replace them with stubs and where a separate supplier test environment is needed.
• Contacts for the client side, server side, and infrastructure — one person who consolidates decisions and confirms acceptance.
• Existing artifacts: old tests, failure logs, defect reports, description of the release cycle and deployment windows.

Then we agree on the order: first a quick set of checks for every branch merge, then expansion to full regression on schedule. Describe the task and send what you already have — we will return with an estimate, an E2E testing implementation plan, and responsibility boundaries.