How We Reduced E2E Regression Run Times by 70% Using Parallel Execution

Slow end-to-end suites do not fail loudly. They fail by changing behaviour: developers stop running E2E locally, pull requests wait 30 minutes for a green check, and eventually someone moves the suite to a nightly job where regressions surface a day late.

Most slow Playwright suites are slow for the same four reasons:

• Serial execution. workers: 1 or fullyParallel: false, often added years ago to "fix" a flaky test.
• Repeated sign-in. Every test logs in through the UI, paying for three requests and a redirect before it tests anything.
• Over-recording. trace: "on" or video: "on" for every test, not just failures.
• Naive sharding. Splitting by test count puts all the slow tests on one machine, so the slowest shard sets the wall-clock time.

To put real numbers on each fix, we built a 43-test regression suite against a small demo shop (sign-in, catalogue search, cart, checkout, order history, navigation) and measured every change separately on the same 2-vCPU Linux machine with headless Chromium. Starting at 186.9 seconds, we reached 51.8 seconds with four workers on one machine (72% faster) and 32.1 seconds across two duration-balanced shards (83% faster), with zero failures and zero flaky tests across every run.

Everything below is the code we ran: the Playwright config, the per-worker sign-in fixture, a duration-aware shard planner and a GitHub Actions workflow that runs shards in Docker and merges their reports.

How Playwright parallelism works

Three levels of parallelism

Level 1  fullyParallel         tests inside one file run concurrently
Level 2  workers               N browser processes on one machine
Level 3  shards                M machines, each running its own workers

  CI run
  ├── plan job ───────── reads last run's timings, writes shard-1.txt … shard-M.txt
  ├── shard 1 (runner) ─ worker 1 · worker 2 · worker 3 · worker 4 ─┐
  ├── shard 2 (runner) ─ worker 1 · worker 2 · worker 3 · worker 4 ─┼─▶ blob reports
  ├── …                                                              │
  └── report job ◀──── merge-reports: one HTML report + timings.json ┘
                         └──── timings.json feeds the next plan job

• fullyParallel: true lets Playwright distribute individual tests, not whole files. Without it, one slow file pins one worker while the others sit idle.
• Workers are separate processes, each with its own browser. E2E tests spend most of their time waiting on the network and the app, so you can run more workers than CPU cores. In our runs, 4 workers on 2 vCPUs was still a large gain.
• Shards split the suite across machines. Playwright's built-in --shard=1/4 splits by test count. That is only fair when tests take similar time, and they rarely do.

Test isolation is the price of parallelism

Parallel tests break when they share state. In our demo shop, every signed-in user has one cart. If all workers sign in as the same user, one test's "add to cart" shows up in another test's total. The rule that makes parallelism safe is:

• One account per worker, created or signed in once, reused by every test that worker runs.
• A reset before each test, so test order inside a worker does not matter.

Dynamic load balancing

Count-based sharding ignores duration. In our suite, the 8 order-history tests (up to 8 checkouts each) take far longer than the 8 navigation tests. With --shard=1/2, Playwright put most of the heavy tests together: one shard took 20.0 s and the other 35.6 s. A planner that reads each test's duration from the previous run and packs tests longest-first into the least-loaded shard brought it to 32.1 s and 29.3 s.

Making the suite parallel, step by step

Step 1: The "before" baseline

This is the pattern most slow suites start from: every test signs in through the UI.

// before/sign-in.ts — called at the start of every test
import { expect, type Page } from "@playwright/test";

/** The "before" suite: every test signs in through the UI first. */
export async function signIn(page: Page) {
  await page.goto("/login");
  await page.getByLabel("Email").fill("someone@example.test");
  await page.getByLabel("Password").fill("correct-horse");
  await page.getByRole("button", { name: "Sign in" }).click();
  await expect(page).toHaveURL(/\/products$/);
}

Step 2: Configure Playwright for parallel runs

// playwright.config.ts
import { defineConfig, devices } from "@playwright/test";

const isCI = !!process.env.CI;

export default defineConfig({
  testDir: "./e2e",
  // Spread individual tests, not just files, across workers and shards.
  fullyParallel: true,
  forbidOnly: isCI,
  retries: isCI ? 1 : 0,
  // Size workers to the runner: 2-vCPU GitHub runners handle ~4 I/O-bound browser workers.
  workers: process.env.PW_WORKERS ? Number(process.env.PW_WORKERS) : isCI ? 4 : "50%",
  // Blob reports from every shard are merged into one HTML report afterwards.
  reporter: isCI ? [["blob"], ["github"]] : [["list"], ["html", { open: "never" }]],
  use: {
    baseURL: process.env.BASE_URL ?? "http://localhost:4173",
    // Traces are expensive to record; keep them for the retry that explains a failure.
    trace: "on-first-retry",
    screenshot: "only-on-failure",
    video: "off",
  },
  projects: [{ name: "chromium", use: { ...devices["Desktop Chrome"] } }],
  webServer: {
    command: "node app/server.mjs",
    url: "http://localhost:4173/login",
    reuseExistingServer: !isCI,
    timeout: 60_000,
  },
});

Key decisions:

• fullyParallel: true for test-level distribution.
• trace: "on-first-retry" instead of "on". You get a full trace exactly when a test fails once, without paying for one on every pass.
• retries: 1 in CI only. A test that passes on retry is reported as flaky, which is a signal to fix, not to ignore.
• A blob reporter in CI, so reports from every shard can be merged into one.

Step 3: Sign in once per worker, not once per test

A worker-scoped fixture signs in through the API (no browser, no UI), saves the cookies to a storage-state file named after the worker, and hands that file to every test the worker runs. An auto fixture resets the account's cart before each test.

// e2e/fixtures.ts
import { test as base, expect, request } from "@playwright/test";
import fs from "node:fs";
import path from "node:path";

// One account (and one signed-in storage state) per parallel worker: tests in the same worker
// share it, tests in different workers never touch each other's cart or orders.
export const test = base.extend<{ resetState: void }, { workerStorageState: string }>({
  storageState: ({ workerStorageState }, use) => use(workerStorageState),

  workerStorageState: [
    async ({}, use, workerInfo) => {
      const file = path.resolve(workerInfo.project.outputDir, `.auth/worker-${workerInfo.parallelIndex}.json`);
      if (!fs.existsSync(file)) {
        const api = await request.newContext({ baseURL: workerInfo.project.use.baseURL });
        const res = await api.post("/api/login", {
          data: { email: `e2e+w${workerInfo.parallelIndex}@example.test`, password: process.env.E2E_PASSWORD ?? "correct-horse" },
        });
        expect(res.ok()).toBeTruthy();
        fs.mkdirSync(path.dirname(file), { recursive: true });
        await api.storageState({ path: file });
        await api.dispose();
      }
      await use(file);
    },
    { scope: "worker" },
  ],

  // Every test starts from an empty cart, whatever ran before it in this worker.
  resetState: [
    async ({ page }, use) => {
      const res = await page.request.post("/api/reset");
      expect(res.ok()).toBeTruthy();
      await use();
    },
    { auto: true },
  ],
});

export { expect };

Tests import test and expect from the fixture file and start already signed in:

// e2e/checkout.spec.ts
import { test, expect } from "./fixtures";

test.describe("checkout", () => {
  for (const item of [1, 3, 5, 7, 9, 11, 13, 15]) {
    test(`checks out product ${item} and confirms the order`, async ({ page }) => {
      await page.goto("/products");
      await page.getByRole("listitem").filter({ hasText: new RegExp(`^Product ${item} `) }).getByRole("button").click();
      await expect(page.getByTestId("cart-count")).toHaveText("1");
      await page.goto("/cart");
      await expect(page.getByTestId("cart-item")).toHaveText(`Product ${item}`);
      await page.getByRole("button", { name: "Checkout" }).click();
      await expect(page.getByRole("status")).toHaveText(/Order \d+ confirmed/);
      await page.goto("/orders");
      await expect(page.getByTestId("order")).toHaveCount(1);
    });
  }
});

Keep a separate spec that tests sign-in itself through the UI, using the plain @playwright/test import so it starts signed out:

// e2e/auth.spec.ts
import { test, expect } from "@playwright/test";

// Sign-in itself is tested through the UI, without the shared signed-in state.
test.describe("sign-in", () => {
  test("rejects a wrong password", async ({ page }) => {
    await page.goto("/login");
    await page.getByLabel("Email").fill("someone@example.test");
    await page.getByLabel("Password").fill("wrong");
    await page.getByRole("button", { name: "Sign in" }).click();
    await expect(page.getByRole("alert")).toHaveText("Invalid email or password");
  });

  test("signs in and lands on products", async ({ page }) => {
    await page.goto("/login");
    await page.getByLabel("Email").fill("someone@example.test");
    await page.getByLabel("Password").fill(process.env.E2E_PASSWORD ?? "correct-horse");
    await page.getByRole("button", { name: "Sign in" }).click();
    await expect(page).toHaveURL(/\/products$/);
  });

  test("redirects signed-out visitors to sign-in", async ({ page }) => {
    await page.goto("/cart");
    await expect(page).toHaveURL(/\/login$/);
  });
});

Step 4: Plan shards by duration, not by count

The planner reads two Playwright JSON reports: the current list of tests (--list --reporter=json, so new tests are included and deleted tests dropped) and the timings from the last full run. It assigns tests longest-first to the least-loaded shard (the LPT heuristic), and gives tests with no history the median known duration. The output is one file per shard for Playwright's --test-list flag.

// scripts/plan-shards.mjs
// Duration-aware shard planner.
//   node scripts/plan-shards.mjs <shards> <current-tests.json> [timings.json] [outDir]
// <current-tests.json>: `npx playwright test --list --reporter=json` (what exists now)
// [timings.json]:       a JSON report from a previous full run (how long each test took)
// Writes <outDir>/shard-<n>.txt files for `npx playwright test --test-list <file>`.
import fs from "node:fs";
import path from "node:path";

const [shardsArg, currentFile, timingsFile, outDir = "shards"] = process.argv.slice(2);
const shards = Number(shardsArg);
if (!Number.isInteger(shards) || shards < 1 || !currentFile) {
  console.error("usage: plan-shards.mjs <shards> <current-tests.json> [timings.json] [outDir]");
  process.exit(2);
}

/** Flattens a Playwright JSON report into { key, line, ms } entries, one per test per project. */
function collect(report) {
  const out = [];
  const walk = (suite, titles) => {
    const here = suite.title && !suite.file?.endsWith(suite.title) ? [...titles, suite.title] : titles;
    for (const spec of suite.specs ?? []) {
      for (const test of spec.tests) {
        const titlePath = [...here, spec.title];
        const line = `[${test.projectName}] › ${spec.file} › ${titlePath.join(" › ")}`;
        const ms = test.results?.reduce((sum, r) => sum + r.duration, 0) ?? 0;
        out.push({ key: line, line, ms });
      }
    }
    for (const child of suite.suites ?? []) walk(child, here);
  };
  for (const fileSuite of report.suites ?? []) walk(fileSuite, []);
  return out;
}

const current = collect(JSON.parse(fs.readFileSync(currentFile, "utf8")));
const history = new Map(
  timingsFile && fs.existsSync(timingsFile)
    ? collect(JSON.parse(fs.readFileSync(timingsFile, "utf8"))).filter((t) => t.ms > 0).map((t) => [t.key, t.ms])
    : [],
);

// New tests have no history yet: assume the median known duration (or 10 s with no history at all).
const known = [...history.values()].sort((a, b) => a - b);
const fallback = known.length ? known[Math.floor(known.length / 2)] : 10_000;
const tests = current.map((t) => ({ line: t.line, ms: history.get(t.key) ?? fallback }));

// Longest-processing-time-first: hand each test, slowest first, to the least-loaded shard.
tests.sort((a, b) => b.ms - a.ms);
const bins = Array.from({ length: shards }, () => ({ ms: 0, lines: [] }));
for (const t of tests) {
  const bin = bins.reduce((min, b) => (b.ms < min.ms ? b : min));
  bin.ms += t.ms;
  bin.lines.push(t.line);
}

fs.mkdirSync(outDir, { recursive: true });
bins.forEach((bin, i) => fs.writeFileSync(path.join(outDir, `shard-${i + 1}.txt`), bin.lines.join("\n") + "\n"));
for (const [i, bin] of bins.entries()) {
  console.log(`shard ${i + 1}: ${bin.lines.length} tests, ~${(bin.ms / 1000).toFixed(1)} s of test time`);
}

Run it locally to see the balance it produces:

npx playwright test --list --reporter=json > current.json
node scripts/plan-shards.mjs 2 current.json timings.json shards
npx playwright test --test-list shards/shard-1.txt

--test-list matches lines in the form [project] › file › describe › test title, which is exactly what the planner writes. If you rename a Playwright project, the next plan picks it up automatically from the fresh --list output.

Step 5: Run shards in Docker on GitHub Actions

Three jobs: plan builds the shard files from cached timings, e2e runs each shard in the official Playwright container, and report merges every shard's blob report into one HTML report plus a timings.json that is cached for the next plan. Timings are saved only from main, so pull-request branches never pollute them.

# .github/workflows/e2e.yml
name: E2E

on:
  pull_request:
  push:
    branches: [main]

concurrency:
  group: e2e-${{ github.ref }}
  cancel-in-progress: true

env:
  SHARDS: 4 # keep in sync with the matrix below

jobs:
  plan:
    runs-on: ubuntu-latest
    # The official image ships browsers and OS dependencies; pin it to your @playwright/test version.
    container: { image: "mcr.microsoft.com/playwright:v1.56.0-noble", options: "--user 1001" }
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }
      - run: npm ci
      - name: Restore test timings from the last main run
        uses: actions/cache/restore@v4
        with:
          path: timings.json
          key: pw-timings-${{ github.run_id }}
          restore-keys: pw-timings-
      - name: Plan shards by duration
        run: |
          npx playwright test --list --reporter=json > current.json
          node scripts/plan-shards.mjs "$SHARDS" current.json timings.json shards
      - uses: actions/upload-artifact@v4
        with: { name: shard-plan, path: shards/, retention-days: 1 }

  e2e:
    needs: plan
    runs-on: ubuntu-latest
    timeout-minutes: 20
    container: { image: "mcr.microsoft.com/playwright:v1.56.0-noble", options: "--user 1001 --ipc=host" }
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    env:
      CI: "true"
      E2E_PASSWORD: ${{ secrets.E2E_PASSWORD }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }
      - run: npm ci
      - uses: actions/download-artifact@v4
        with: { name: shard-plan, path: shards }
      - name: Run shard ${{ matrix.shard }} of ${{ env.SHARDS }}
        run: npx playwright test --test-list shards/shard-${{ matrix.shard }}.txt --reporter=blob,github
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: blob-report-${{ matrix.shard }}
          path: blob-report/
          retention-days: 1

  report:
    needs: e2e
    if: ${{ !cancelled() }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }
      - run: npm ci
      - uses: actions/download-artifact@v4
        with: { pattern: blob-report-*, path: all-blob-reports, merge-multiple: true }
      - name: Merge shard reports into one HTML report and one timings file
        run: npx playwright merge-reports --reporter html,json ./all-blob-reports
        env:
          PLAYWRIGHT_JSON_OUTPUT_NAME: timings.json
      - uses: actions/upload-artifact@v4
        with: { name: playwright-report, path: playwright-report/, retention-days: 14 }
      - name: Save timings for the next shard plan
        if: github.ref == 'refs/heads/main'
        uses: actions/cache/save@v4
        with:
          path: timings.json
          key: pw-timings-${{ github.run_id }}

To reproduce a CI failure locally in the same environment:

docker run --rm --ipc=host -v "$PWD":/work -w /work \
  mcr.microsoft.com/playwright:v1.56.0-noble \
  bash -c "npm ci && npx playwright test --test-list shards/shard-2.txt"

Pin the image tag to the same version as @playwright/test in your lockfile. On a mismatch, Playwright fails at launch because the browser build it expects is not in the image.

What each change saved: measured results

How we measured

• Suite: 43 Playwright tests across 6 spec files (sign-in, catalogue search, cart, checkout, order history, navigation) against a Node demo shop whose API responds with fixed, realistic delays (200–450 ms per call).
• Machine: one 2-vCPU Linux container with 7 GB of RAM, headless Chromium, Playwright 1.56.
• Method: one change at a time, same machine, same app. The 4-worker configuration was run three times to check variance (51.5 s, 52.2 s, 51.7 s; mean 51.8 s). Shard runs were executed one after the other on the same machine and recorded per shard; in CI they run on separate runners at the same time.

Run times for each configuration

All 43 tests passed in every configuration, with no flaky results across ten full or sharded runs.

What the numbers show

• Workers are the biggest single lever. Going from 1 to 4 workers cut the time by two-thirds on a machine with only 2 vCPUs, because browser tests mostly wait on I/O.
• Returns diminish. Going from 4 to 6 workers saved another 10.6 seconds, far less than the first steps did. Once the CPU is saturated, extra workers make every action slower and timing-sensitive tests flakier, so benchmark your own runner size before raising the count.
• Sign-in reuse is a free 10%+. It also removes the most common source of flakiness from every test except the ones that test sign-in itself.
• Count-based shards waste capacity. The two count-based shards took 20.0 s and 35.6 s: one runner sat idle for 15.6 seconds. Duration-based shards took 32.1 s and 29.3 s, a 2.8-second gap. On a suite that runs for 20 minutes, that same imbalance is minutes per run, on every pull request.

Cost model

GitHub Actions bills each job's minutes separately, and each runner pays a fixed setup cost (checkout, container pull, npm ci) before the first test starts. That gives a simple model for a test time of T minutes, a setup cost of S minutes and N shards:

Sharding always costs more total minutes, by roughly (N − 1) × S. It buys back wall-clock time when T is large relative to S. Measure S in your own workflow logs: for a suite that runs 3 minutes on one runner, sharding rarely pays; for one that runs 25 minutes, four shards turn a coffee break into a short wait.

Mistakes that slow parallel suites down

• Sharing one account across workers. The suite passes with workers: 1 and fails randomly with workers: 4. Use one account per worker (parallelIndex), and reset state before each test.
• Raising workers to "fix" slowness on a small runner. Past the point of CPU saturation, every action slows down and timeouts appear. Raise workers in steps and watch for flaky tests, not just total time.
• fullyParallel: false left over from an old workaround. If one file has order-dependent tests, mark only that file with test.describe.configure({ mode: "serial" }) and parallelise everything else.
• trace: "on" and video: "on" everywhere. Recording costs time and storage on every pass. Use on-first-retry and only-on-failure.
• Timings saved from pull requests. A PR that slows one test down should not reshape the shard plan for everyone else. Save timings from main only.
• Matrix and shard count drifting apart. If SHARDS says 4 and the matrix lists 3 entries, a quarter of the suite silently never runs. Keep them side by side, or generate the matrix from the plan job's output.
• Retries hiding real flakiness. Track tests reported as flaky and fix them weekly. Consider --fail-on-flaky-tests on main once the backlog is clear.
• Docker image and package versions out of sync. Pin mcr.microsoft.com/playwright:vX.Y.Z to the exact @playwright/test version and update both in the same pull request.
• Merging reports from a different Playwright version. merge-reports needs the blob reports and the merging job on the same version; install from the same lockfile in every job.

Your parallel testing checklist

• Measure your baseline: total run time, slowest 10 tests, flaky count
• Set fullyParallel: true and remove blanket serial modes
• Replace per-test UI sign-in with a worker-scoped API sign-in fixture and a per-test reset
• Switch traces to on-first-retry and screenshots to only-on-failure
• Find the worker count that is fastest on your CI runner without new flaky tests
• Add blob reporting and merge-reports so sharded runs still produce one report
• Add the duration-aware shard planner and cache timings from main
• Run shards in the pinned Playwright Docker image, with --ipc=host
• Re-measure, and record the before-and-after in your pull request

If your regression suite has outgrown your release cadence, our test automation team builds and tunes Playwright suites like this one, and our regression testing service keeps them fast and trustworthy as your product grows. For choosing a framework in the first place, read our Playwright vs Cypress vs Selenium comparison.

Frequently asked questions

How many Playwright workers should I use in CI? — Benchmark it on your runner. In our measurements on a 2-vCPU machine, 4 workers cut the suite from 186.9 s to 51.8 s and 6 workers to 41.2 s; raise the count in steps and stop when new flaky tests appear.

What is the difference between Playwright workers and shards? — Workers are parallel browser processes on one machine. Shards split the suite across several machines, each running its own workers, and their blob reports are merged into one report afterwards.

Why are my Playwright shards uneven? — The built-in --shard option splits by test count, not duration, so slow tests can cluster on one machine. Planning shards from the previous run's timings with --test-list balances them.

Does sharding reduce CI cost? — No, it usually increases billed minutes, because every shard pays its own setup time. It reduces wall-clock time, so it pays off when test time is large compared with runner setup.