Optimizing Test Execution: How We Reduced a Large Automation Suite from 8 Hours to 1.5 Hours

Testing a large, data-heavy web application is not only about writing automated test scripts. At scale, test execution becomes a system-design challenge. The biggest problems usually come from application state, shared data, workflow dependencies, test cleanup, and the structure of the test framework.

When we started validating a complex enterprise web application with more than 600 test cases, our biggest challenge was execution time. A full validation cycle took more than eight hours. That meant slow feedback, delayed debugging, and limited confidence during active development.

After several iterations, we reduced the full execution cycle to approximately 1 hour and 25 minutes, with all 623 test cases passing consistently.

This article explains how we moved from slow, non-deterministic execution to a stable Playwright-based automation framework. It covers the challenges we faced, the architectural changes we made, the concurrency problems we solved, and the lessons we learned while building a maintainable end-to-end test suite.

The starting point: a slow and unstable execution cycle

The application under test was a complex internal business portal with multiple pages, tabs, tables, dynamic fields, save actions, reload behavior, and cross-section data dependencies.

The test suite needed to validate several types of behavior:

• Basic UI field visibility
• Field-level input validation
• Save and reload persistence
• Payload-level verification
• Cross-section data propagation
• Dynamic table behavior
• Conditional field rendering
• Master-data mapping
• Workflow-level state transitions

At the beginning, the suite contained more than 600 test cases. The problem was not just the number of tests. The real issue was that many scenarios depended on shared application state.

A single test could create or update a record, another could verify whether the saved value persisted, and another could validate whether the same value appeared correctly in a different section of the application.

Because of these dependencies, execution was slow and difficult to parallelize safely.

The original execution profile looked like this:

Total test cases: 600+
Execution time: 8+ hours
Result: Slow feedback and inconsistent debugging cycles

An eight-hour feedback loop was too long for active engineering. If a failure occurred, we had to inspect the result, adjust the script or data, and rerun a large part of the suite. This made every debugging cycle expensive.

Our goals were clear:

• Reduce execution time
• Improve stability
• Remove flaky behavior
• Create a repeatable automation framework
• Keep the test suite maintainable

Phase 1: initial browser-agent-based execution

Our first automation approach used browser-based agents and semi-automated execution flows. This helped us quickly validate whether the application could be tested through the UI, but it was not fast enough for large-scale execution.

The initial throughput was approximately:

125 test cases / 5 hours

This was not sustainable. At that rate, completing the full suite required a very long execution window.

The agent-based approach had another issue: execution behavior was not fully deterministic. The same scenario could behave differently across runs because the agent had to interpret the UI, wait for dynamic elements, decide the next interaction, and handle state transitions.

For smaller batches, this was manageable. For more than 600 tests, it became a bottleneck.

Attempting to improve throughput with parallel agents

To improve speed, we experimented with running multiple agents in parallel. The idea was to divide the work across several browser sessions. One agent could create records, another could fill fields, another could save changes, and another could verify output values.

A simplified flow looked like this:

Agent 1: Create or update a record
Agent 2: Enter field values
Agent 3: Save and reload the page
Agent 4: Verify expected values

We also tried combining related validations into a single compound execution flow:

Create row
  → Fill multiple fields
  → Save changes
  → Reload page
  → Verify field values
  → Verify related section behavior

This reduced repeated navigation and allowed one execution flow to cover multiple test cases. However, when we scaled the approach, we started seeing serious problems.

Problem 1: shared-state interference

The application was stateful. Many tests interacted with the same parent record, table, section, or workflow state. When multiple agents ran at the same time, they interfered with one another.

A typical failure pattern looked like this:

Agent A creates a row
Agent B edits the parent record
Agent C reloads a related section
Agent D modifies or deletes dependent data

Each agent was performing a valid action in isolation. Together, however, they were corrupting the assumptions of other running tests.

This created failures that were hard to diagnose. A verification could fail in one section because another parallel session had changed the underlying data a few seconds earlier.

These were not real application defects. They were automation-induced failures caused by unsafe concurrency.

Problem 2: inconsistent results

Parallel agent execution improved throughput slightly, but consistency became worse. The same test could pass in one run and fail in another without any code change.

At this stage, throughput improved to approximately:

150–200 test cases / 5 hours

This was better than the original execution model, but it was still too slow and too flaky.

A test suite is only useful when engineers can trust the result. If every failed test requires manual investigation to determine whether the issue is real or caused by automation behavior, the suite becomes a burden instead of a confidence-building tool.

Introducing the 1-writer / multi-reader pattern

To reduce state conflicts, we introduced a controlled execution model called the 1-writer / multi-reader pattern.

The concept was simple: only one execution flow was allowed to mutate data. Multiple other execution flows could only read and verify data.

The model looked like this:

Writer:
  Creates records
  Updates fields
  Saves data
  Changes workflow state

Readers:
  Verify UI values
  Verify reload behavior
  Verify related-section output
  Verify read-only expectations

This improved stability because only one session was responsible for changing state. However, it did not fully solve the problem. The writer became the bottleneck, reader sessions often had to wait, and execution was still not deterministic enough.

The improved throughput looked like this:

200–300 test cases / 4 hours

This was progress, but not enough. A full execution cycle was still close to eight hours. At this point, we realized the core issue was the execution model itself. We needed to move away from semi-deterministic agent execution and build a dedicated automation framework.

Phase 2: moving to deterministic Playwright automation

The next major step was to rebuild the automation suite using Playwright.

Playwright gave us direct control over browser behavior, selectors, assertions, retries, traces, screenshots, and test execution. Instead of relying on an agent to interpret the page, we could define the exact steps required for each test.

A simple Playwright test looked like this:

import { test, expect } from '@playwright/test';

test('should persist a field value after save and reload', async ({ page }) => {
  await page.goto('/app/module');

  await page.getByRole('button', { name: 'Add Row' }).click();
  await page.getByLabel('Record Name').fill('AUTO_TEST_RECORD');
  await page.getByRole('button', { name: 'Save' }).click();

  await expect(page.getByText('Saved successfully')).toBeVisible();

  await page.reload();

  await expect(page.getByLabel('Record Name')).toHaveValue('AUTO_TEST_RECORD');
});

This approach was much more predictable. We could control navigation, assertions, waits, and cleanup.

However, the first Playwright version was still not perfect. The initial scripts were generated too closely from raw test-case documents. They followed the written steps, but they did not fully account for how the application behaved during real execution.

For example, a test case might say:

Open section
Enter value
Save
Verify value

But the real application flow required more context:

Open module
Wait for page data to load
Select the correct section
Wait for dynamic table rendering
Select or create the required row
Wait for conditional fields
Enter value
Wait for validation or debounce behavior
Save
Wait for success confirmation
Reload
Restore the same context
Verify persisted value

This difference was important. If automation follows only the written test steps and ignores real application behavior, the tests become brittle.

The early Playwright version reduced execution time to approximately 3–4 hours. That was a major improvement, but we still had flaky failures. The scripts were faster, but the framework needed better structure.

Phase 3: restructuring the test source of truth

The biggest improvement came when we stopped treating all 600+ tests as one large flat suite. Instead, we classified the tests into domain categories based on what they were actually validating.

We grouped the test cases into five broad categories:

1. UI-only scenarios
2. UI and payload validations
3. Workflow persistence
4. Cross-section data flow
5. Master-data source-of-truth mapping

This structure made the suite much easier to reason about. A UI-only test did not need the same setup as a persistence test. A payload validation did not need the same execution path as a cross-section workflow test. A master-data mapping test required a different level of setup and verification.

Once we created this classification, we were able to design the framework around test intent instead of treating every test the same way.

Building a manifest-driven framework

We introduced a unified manifest file that described each test case using metadata.

Each test case included information such as:

• Test ID
• Title
• Domain category
• Module name
• Section name
• Table name
• Field name
• Input value
• Expected value
• Whether save was required
• Whether reload was required
• Whether the test depended on another section

A simplified manifest structure looked like this:

export type TestDomain =
  | 'ui'
  | 'payload'
  | 'persistence'
  | 'cross-section'
  | 'master-data';

export type TestCaseManifest = {
  id: string;
  title: string;
  domain: TestDomain;
  module: string;
  section: string;
  table?: string;
  field: string;
  inputValue?: string;
  expectedValue: string;
  requiresSave: boolean;
  requiresReload: boolean;
  dependsOn?: string[];
};

export const testCases: TestCaseManifest[] = [
  {
    id: 'TC-001',
    title: 'Field value should persist after save and reload',
    domain: 'persistence',
    module: 'Main Module',
    section: 'Details Section',
    table: 'Primary Table',
    field: 'Record Name',
    inputValue: 'AUTO_TC_001',
    expectedValue: 'AUTO_TC_001',
    requiresSave: true,
    requiresReload: true
  }
];

This gave us a clean separation between test data and execution logic. Instead of writing hundreds of separate test files with repeated navigation and validation code, we could generate tests from the manifest.

The manifest runner

The manifest runner became the heart of the framework. It loaded each test case, selected the right execution strategy, performed the required actions, and verified the expected result.

import { test, expect } from '@playwright/test';
import { testCases } from './manifest';
import { AppPage } from './pages/AppPage';

for (const testCase of testCases) {
  test(`${testCase.id}: ${testCase.title}`, async ({ page }) => {
    const app = new AppPage(page);

    await app.openModule(testCase.module);
    await app.openSection(testCase.section);

    if (testCase.table) {
      await app.openTable(testCase.table);
    }

    if (testCase.inputValue) {
      await app.fillField(testCase.field, testCase.inputValue);
    }

    if (testCase.requiresSave) {
      await app.save();
    }

    if (testCase.requiresReload) {
      await app.reloadAndRestoreContext({
        section: testCase.section,
        table: testCase.table
      });
    }

    await expect(app.getField(testCase.field)).toHaveValue(
      testCase.expectedValue
    );
  });
}

The framework provided several benefits:

• Test logic became reusable.
• New tests could be added through metadata.
• Failures were easier to group by domain.
• Repeated navigation logic moved into page objects.
• Execution became more predictable.
• Debugging became faster.

Most importantly, the framework became easier to maintain. When application navigation changed, we updated the page-object layer instead of modifying hundreds of individual test cases.

Designing stable page objects

Page objects were a key part of the framework.

The first page objects were too generic. They assumed all sections, tables, and fields behaved the same way. In reality, some sections rendered immediately, some loaded after data hydration, some fields appeared conditionally, and some tables required row selection before inputs became available.

We redesigned the page objects to reflect actual application behavior.

import { Page, Locator, expect } from '@playwright/test';

export class AppPage {
  constructor(private readonly page: Page) {}

  async openModule(moduleName: string) {
    await this.page.getByRole('navigation').getByText(moduleName).click();
    await expect(
      this.page.getByRole('heading', { name: moduleName })
    ).toBeVisible();
  }

  async openSection(sectionName: string) {
    await this.page.getByRole('tab', { name: sectionName }).click();
    await expect(
      this.page.getByRole('region', { name: sectionName })
    ).toBeVisible();
  }

  async openTable(tableName: string) {
    await expect(
      this.page.getByRole('table', { name: tableName })
    ).toBeVisible();
  }

  getField(fieldName: string): Locator {
    return this.page.getByLabel(fieldName).or(
      this.page.getByPlaceholder(fieldName)
    );
  }

  async fillField(fieldName: string, value: string) {
    const field = this.getField(fieldName);

    await expect(field).toBeVisible();
    await field.fill(value);
    await expect(field).toHaveValue(value);
  }

  async save() {
    await this.page.getByRole('button', { name: 'Save' }).click();
    await expect(this.page.getByText('Saved successfully')).toBeVisible();
  }

  async reloadAndRestoreContext(context: {
    section: string;
    table?: string;
  }) {
    await this.page.reload();
    await this.page.waitForLoadState('networkidle');

    await this.openSection(context.section);

    if (context.table) {
      await this.openTable(context.table);
    }
  }
}

The goal was to avoid fragile selectors and arbitrary waits.

Instead of relying on CSS paths like this:

await page.locator('.container > div:nth-child(3) input').fill('AUTO_VALUE');

we used semantic locators wherever possible:

await page.getByLabel('Record Name').fill('AUTO_VALUE');

And instead of fixed sleeps like this:

await page.waitForTimeout(3000);

we used deterministic expectations:

await expect(page.getByText('Saved successfully')).toBeVisible();
await expect(page.getByRole('table', { name: 'Primary Table' })).toBeVisible();

This made the tests faster and more reliable.

Phase 4: debugging the remaining failures

After restructuring the source of truth and moving to the manifest-driven Playwright framework, execution time dropped sharply. At one point, the suite could complete much faster than before, but it surfaced around 80 failures.

That initially looked concerning, but it was actually a useful milestone. The failures were no longer random. They were consistent, repeatable, and easier to classify.

We grouped the failures into three root causes:

1. UI alignment gaps
2. Dirty state between tests
3. Worker race conditions

Each category required a different fix.

Root cause 1: UI alignment gaps

Some tests were failing because the automation did not match the live application flow closely enough. For example, a script might try to fill a field before the required table was fully rendered, or validate a value before a background save operation completed.

The fix was not to add random waits. The fix was to wait for meaningful application states.

Instead of this:

await page.getByLabel('Amount').fill('1000');

we used this:

await page.getByRole('tab', { name: 'Details Section' }).click();

await expect(
  page.getByRole('table', { name: 'Primary Table' })
).toBeVisible();

const amountField = page.getByLabel('Amount');

await expect(amountField).toBeVisible();
await amountField.fill('1000');
await expect(amountField).toHaveValue('1000');

This ensured that the test interacted with the application only after the correct UI state was available.

Root cause 2: dirty state between tests

Some tests left behind data that affected later tests. For example, a test might create a row, update a value, or change a workflow state. If that data was not cleaned up properly, another test could start with unexpected state.

This caused false failures.

To solve this, we introduced test-data isolation and cleanup. Each automation-created record used a predictable prefix:

function buildTestRecordName(testId: string) {
  return `AUTO_${testId}_${Date.now()}`;
}

We also added cleanup hooks:

import { test } from '@playwright/test';
import { AppPage } from './pages/AppPage';

test.afterEach(async ({ page }, testInfo) => {
  const app = new AppPage(page);

  try {
    await app.cleanupTestData(testInfo.title);
  } catch (error) {
    console.warn(`Cleanup failed for ${testInfo.title}`, error);
  }
});

The cleanup method removed only automation-created data:

async cleanupTestData(testTitle: string) {
  const testId = testTitle.split(':')[0];
  const testDataPrefix = `AUTO_${testId}`;

  await this.page.goto('/app/test-data');

  await this.page.getByPlaceholder('Search').fill(testDataPrefix);

  const matchingRows = this.page.getByRole('row').filter({
    hasText: testDataPrefix
  });

  while (await matchingRows.count()) {
    await matchingRows.first().getByRole('button', { name: 'Delete' }).click();
    await this.page.getByRole('button', { name: 'Confirm' }).click();
  }
}

This made reruns much safer. A failed test no longer polluted the next execution cycle.

Root cause 3: worker race conditions

The most important stability issue came from worker-level parallelism.

Playwright supports parallel execution using multiple workers. This is excellent for isolated, stateless tests. However, our suite contained many stateful workflows. Several tests depended on shared records, related sections, and ordered data transitions.

When multiple workers ran at the same time, they created race conditions:

Worker A updates a row
Worker B modifies the parent record
Worker C reloads a related section
Worker D deletes temporary test data

The solution was to force single-worker execution:

npx playwright test --workers=1

We also made this part of the Playwright configuration:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  workers: 1,
  retries: 0,
  reporter: [
    ['list'],
    ['html', { outputFolder: 'playwright-report' }]
  ],
  use: {
    baseURL: process.env.BASE_URL,
    trace: 'retain-on-failure',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure'
  }
});

At first, reducing parallelism felt like a performance trade-off. In practice, it improved the overall feedback cycle.

A fast but flaky 30-minute run was less useful than a stable 1.5-hour run. With a deterministic single-worker run, failures became meaningful. Engineers could trust that a failure represented either a real application issue or a specific framework issue, not a random race condition.

The final execution model

The final test execution model was simple and reliable:

Load test manifest
  → Group test cases by validation type
  → Open the required module
  → Navigate to the correct section
  → Prepare test data
  → Execute deterministic UI actions
  → Save only when required
  → Reload only when required
  → Restore application context
  → Verify expected result
  → Clean up automation-created data
  → Continue with the next test

This gave us the right balance between speed, stability, and maintainability.

Final results:

Total test cases: 623
Passing test cases: 623
Execution time: 1 hour 25 minutes
Execution mode: Single worker
Result: Stable and repeatable

Key takeaways

1. Deterministic state is more important than maximum parallelism

Parallel execution is useful when tests are isolated. But when tests share application state, uncontrolled parallelism can create race conditions.

In our case, running everything through one worker produced a more trustworthy result. The suite became slightly slower than aggressive parallel execution, but much faster than repeatedly debugging flaky failures.

The goal of automation is not just speed. The goal is confidence.

2. A good test suite needs a clear data architecture

The biggest improvement came from restructuring the source of truth.

Once we categorized test cases into UI-only, payload, persistence, cross-section, and master-data groups, the framework became much easier to maintain.

Each test had a clear purpose. Each validation type had a clear execution strategy. That structure made the suite scalable.

3. Page objects should reflect real application behavior

Page objects should not simply mirror a test specification document. They should reflect how the application actually behaves.

That means handling:

• Dynamic loading
• Conditional rendering
• Save confirmation
• Reload behavior
• Context restoration
• Table hydration
• Field visibility
• Workflow transitions

When page objects understand real application behavior, tests become less fragile.

4. Cleanup is part of automation design

For stateful applications, cleanup cannot be an afterthought.

Every test should either create isolated data or restore the application to a known state. Without cleanup, one passing test can cause another test to fail later.

Reliable automation requires predictable setup and predictable teardown.

5. A stable 1.5-hour run is better than a flaky 30-minute run

Execution time matters, but trust matters more.

A flaky fast run creates confusion. A deterministic run creates confidence.

Once the suite became stable, the 1 hour and 25 minute runtime was acceptable because the results were clean, repeatable, and actionable.

Before and after

The improvement was significant:

Before:
  Execution time: 8+ hours
  Behavior: Slow and inconsistent
  Debugging: Expensive
  Confidence: Low

After:
  Execution time: 1 hour 25 minutes
  Test cases: 623
  Passing: 623
  Behavior: Stable and deterministic
  Confidence: High

Conclusion

Reducing a large automation suite from more than eight hours to around 1.5 hours required more than switching tools. It required a complete shift in automation strategy.

We moved away from non-deterministic browser-agent execution and built a Playwright-based framework around structured test metadata, stable page objects, controlled state management, and single-worker execution.

The most important lesson was that scalable automation is not just about writing more scripts or running more workers. It is about understanding application state and designing the framework around that state.

Once we treated state management, test data, cleanup, and execution order as first-class parts of the automation architecture, the suite became faster, cleaner, and more reliable.

The final outcome was a stable test suite that executed 623 test cases in 1 hour and 25 minutes, with consistent, repeatable results.

That is the difference between automation that slows a team down and automation that gives a team confidence to move faster.