What this is: A field guide, not a tutorial. These are principles distilled from real-world scars — late nights, flaky tests, and hard-won lessons. Beginners welcome. Battle-hardened veterans, you'll nod along.
The Golden Rules
1. data-testid or Die Trying
Every element that does something (actions), takes something (inputs), or shows something (outputs) needs a data-testid. No exceptions. No "we'll add it later." No "the CSS selector works fine."
Future you will thank present you. Present you should also apologize to past you for all those div > span:nth-child(3) selectors.
2. Build Harnesses, Not Novels
Your test scripts should read like a story, not a dissertation. If your test file is doing HTTP setup, data seeding, authentication flows, AND the actual test — you've written a monolith, not a test.
Build the harness in and around your application. The test should just orchestrate.
3. CI Reports Should Be Boring
If your CI report requires a Rosetta Stone to interpret, you've failed. Engineers should glance at it and know:
- What failed?
- Where it failed?
- How to reproduce it?
Nobody has time to scroll through 500 lines of CI logs to find "Expected: true, Received: false".
4. Fast Feedback or No Feedback
A 45-minute CI pipeline is just a very slow way to say "I'll check tomorrow."
If engineers aren't getting feedback fast enough to act on it in the same context switch, you're losing them. Parallelize. Cache. Optimize. Do whatever it takes.
5. You're Building for Humans (Specifically, Tired Ones)
You're not writing tests for the CI. You're writing for the engineer at 4 PM on a Friday who just wants to ship their feature.
- Get feedback from your team
- Iterate based on their pain points
- Document the weird stuff
Growing an automation suite is like growing plants. They need care, pruning, and enough sunlight. Neglected tests become that corner of the garden everyone pretends doesn't exist.
6. No Telemetry = Testing Blind
If you can't answer "how healthy is our test suite right now?" without running the whole thing, you're flying blind.
Track:
- Pass/fail rates over time
- Flakiness scores
- Execution duration trends
You need data to make decisions. Gut feelings don't scale.
7. Quarantine Without Guilt
A flaky test erodes trust. Every time a test fails for "no reason," an engineer loses a little faith. Enough of these and your entire suite becomes background noise.
Quarantine aggressively. Fix methodically. Losing engineering confidence is expensive — earning it back is very expensive.
8. E2E Tests Are Expensive. Act Like It.
Each E2E test is a small commitment to maintain it forever. Before adding one, ask:
- Can this be covered at a lower level?
- Is this a critical user journey?
- Will anyone notice if this breaks?
Treat E2E tests like you're paying for them out of pocket. Because you are. In time.
9. Order Independence Is Non-Negotiable
If your tests only pass when run in a specific order, you don't have tests. You have a very fragile house of cards.
Each test sets up what it needs, cleans up after itself, and makes no assumptions about shared state.
10. Readable Tests > Clever Tests
Use your harnesses to make tests read like specifications. An engineer unfamiliar with the codebase should be able to read a test and understand the user story.
If a test requires archaeological knowledge to understand, refactor it.
11. The Test Shaker: Non-Negotiable
Every new test goes through the shaker before merge. Run it 50 times. Run it 100 times. Find its breaking point.
Flaky tests don't become less flaky in production. They just become your problem later. Catch them at the door.
12. Test What Users See, Not How It's Built
You click. You type. You assert what's visible.
If you're checking internal state, component props, or Redux store contents — that's a unit test, not an E2E test. E2E tests are the user's advocate, not the codebase's internal auditor.
13. Page Object Model: Yes, Still
It's not sexy. It's not new. It's also battle-tested across countless implementations and keeps your tests maintainable.
When the login button moves, you update it in one place. Not forty-seven.
14. Test Data Is Infrastructure
Build tooling for:
- Creating test data
- Cleaning up test data
- Pruning stale test data
If your tests are fighting over the same [email protected], you're going to have a bad time.
15. Determinism Is the Antidote to Flakiness
If your test behaves differently based on:
- Time of day
- What other tests ran before it
- The phase of the moon
You have a determinism problem. Build support into your application:
- Seed control
- Clock manipulation
- Feature flag overrides for tests
Flaky tests are almost always a symptom of non-deterministic behavior.
16. No Fixed Waits. Period.
// If you write this, you owe the team coffee
await page.waitForTimeout(5000);Playwright's locators auto-wait. If you're adding fixed waits, you're treating symptoms, not causes. Find the actual race condition. Fix it. Don't bandaid it.
Fixed waits are a test smell that something deeper is broken.
17. Ephemeral Environments Are Worth It
Yes, they're complex to set up. Yes, they cost more. They're still worth it.
Running tests against an isolated environment per PR:
- Eliminates cross-PR interference
- Increases confidence in results
- Makes debugging dramatically easier
The ROI on this is real.
18. Fixtures Are Your Friends
Playwright fixtures aren't just for browser setup. Build custom fixtures for:
- Test accounts with specific states
- Data cleanup hooks
- Telemetry injection
- Feature flag configurations
A well-designed fixture library is worth its weight in gold.
19. Resist the Wrapper Urge
Do not wrap Playwright's expects, locators or methods "for convenience." You will regret it.
// Don't do this
class CustomPage {
async customClick(selector) {
await this.page.click(selector);
}
}Every wrapper is a maintenance burden. Every abstraction is a layer to debug through. YAGNI (You Ain't Gonna Need It) applies here more than anywhere.
Only abstract when there's proven, repeated pain. Not prophylactic abstraction.
Aspects Worth Adding
20. Traces, Videos, Screenshots: Debug Like a Detective
When a test fails in CI, you're not there to watch it. Capture everything:
- Playwright traces for step-by-step replay
- Videos of failures (not passes — save the storage)
- Screenshots at failure points
Configure these to upload as artifacts. Future you or an engineer debugging will be grateful.
21. Know When to Delete Tests
Tests are not precious. A test that:
- Tests deprecated functionality
- Has been quarantined for 1+ months
- Provides no signal, only noise
…should be deleted. Not "disabled." Deleted. Dead code is dead weight.
22. API Mocking: Pick Your Battles
Sometimes you mock. Sometimes you go real. The decision matrix:
- Mock: External third-party services, rate-limited APIs, services with known instability
- Real: Your own backend when testing integration, critical payment flows
Mock strategically. Going 100% mocked means you're not testing integration. Going 100% real means you're at the mercy of external dependencies.
23. Test Tagging Is Your Friend
Tag your tests by:
- Feature area (@auth, @checkout)
- Priority (@smoke, @regression)
- Stability (@stable, @quarantined)
This enables running subsets of tests, which enables faster feedback loops.
24. Retry Strategies: Handle With Care
Retries can mask flakiness or legitimately handle transient issues. Be intentional:
- Document why a retry exists
- Track retry rates as a metric
- A test that only passes on retry #3 is not passing — it's limping
25. Ownership, Rotation, and Evangelism
Every test should have a clear owner or owning team. Unowned tests become orphans. Orphaned tests rot.
Consider rotation for test maintenance duties. Spreads knowledge, prevents single points of failure, and keeps everyone honest about test quality.
But ownership isn't enough — evangelize. Run lunch-and-learns. Pair with engineers when they break tests. Write a "how to debug a failing test" runbook.
A test suite maintained by one person is a bus factor of one. If you're the only one who understands the suite, you've built a dependency, not a solution. The goal is to make testing everyone's responsibility, not the "automation team's problem."
26. Cross-Browser: Less Than You Think
Here's a hot take: you probably don't need to run everything on Chrome, Firefox, Safari, and Edge.
Most rendering bugs are caught in one browser. Most logic bugs are definitely caught in one. Running your entire suite across 4 browsers is 4x the cost for diminishing returns.
Strategy:
- Chromium: Your daily driver. Run everything here.
- WebKit/Firefox: Run smoke tests and critical paths. Not the full suite.
- Edge: It's Chromium. You're covered.
The exception: if your product explicitly supports a browser and your users are actually on it, test it. Check your analytics. Don't test Safari on principle if 2% of your users are on it.
27. Visual Regression: Handle With Oven Mitts
Visual regression testing sounds great in theory. In practice:
- Screenshots bloat your repo (or your vendor bill)
- Pixel-perfect comparisons break on font rendering differences
- Every minor UI tweak becomes a "update 47 snapshots" commit
Use it sparingly:
- Happy path flows only
- Critical UI that must not change unexpectedly
- Components with complex visual states
This is a scalpel, not a sledgehammer. If you're screenshotting every test, you're going to drown in maintenance.
28. Tiered Execution: Not Everything, Every Time
Your full suite doesn't need to run on every push. That's how you get 45-minute pipelines and engineers making coffee while CI runs.
The pyramid:
- Every push: Critical smoke tests. The "if this breaks, the app is unusable" paths. Fast. Under 5 minutes.
- On release/merge to main: Expanded regression suite. The important journeys.
- Scheduled (nightly/weekly): The full suite. Every edge case. Every browser combo. The works.
This pairs with test tagging (section 23). Tag intentionally, run strategically. Your engineers get fast feedback on pushes, and you still catch edge cases before they hit production.
29. Speak the Same Language as Your App
Testing a React app? Write your tests in TypeScript/JavaScript. Not Java. Not Python.
This isn't about language superiority — it's about cognitive load. Your frontend engineers are already in the JavaScript mental model. Asking them to context-switch to a different language, different tooling, different idioms just to read a test? That's friction. Friction means they won't read the tests. Or fix them. Or write new ones.
Tests are signals for your engineering team. If the people building the app can't easily read, debug, and contribute to the test suite, you've built a silo. Silos become someone else's problem. Then they become everyone's problem.
Match the language. Minimize the context switch.
30. Synthetic Monitoring: Tests That Never Sleep
Your CI tests run on commits. But production doesn't wait for commits to break.
Run your critical smoke tests against production on a schedule — every 5 minutes, every hour, whatever fits. This isn't a replacement for alerting or observability. It's a canary.
If your checkout flow breaks at 3 AM because a third-party payment provider changed something, you want to know before your users do.
This bridges testing and observability. Your tests become early warning systems. The same Playwright scripts you wrote for CI can run against prod with minor configuration changes. Double the value from the same investment.
The Meta Principle
"Tests are production code for your test infrastructure."
Treat them with the same rigor: code review, refactoring, deprecation strategies, documentation.
The moment tests become second-class citizens, the rot begins.
Now go forth and automate. May your selectors be stable and your CI green.
