Smoke vs regression vs E2E tests: which one belongs where?

People use these three terms interchangeably, and most of the time it doesn't matter. But when you're deciding what to actually run on every deploy, the categories matter. They have different jobs, different costs, and different failure modes.

Here's the working definition for each, when to use it, and where it doesn't belong.

Smoke tests

Definition: a quick, narrow check that the system is up and the most basic thing works. The test equivalent of "is the server returning 200 on /health?"

When to use: right after a deploy, before exposing traffic. Smoke tests should run in seconds, not minutes. Their job is to catch deploys that are obviously broken, not to verify product correctness.

Examples for B2B SaaS:

• Hit the homepage, expect 200 with the H1 visible
• Hit the API health endpoint, expect 200
• Load the login page, confirm the form renders
• Database connection check (one cheap query)

Where they don't belong: as your only safety net. A green smoke run tells you the deploy didn't cratter the process. It says nothing about whether checkout works.

Regression tests

Definition: a check that something which used to work still works. The test isn't categorized by what it does, it's categorized by why you wrote it: to catch backslides on functionality you've already shipped.

When to use: on every staging deploy, before promoting to production. The wide-net pass that exercises your critical user paths.

Examples for B2B SaaS:

• User can log in with email and password
• User can create a new project
• The dashboard loads with the right counts after creating data
• Settings page lets the user change their email
• Logout actually logs the user out

Where they don't belong: as a probe into specific function logic. That's a unit test's job. A regression suite should test what the user perceives, not internal implementation.

For more on how regression fits into a B2B SaaS QA stack, see the full regression guide.

E2E tests

Definition: end-to-end tests cover one full user journey, from entry point to outcome, exercising the entire stack (frontend, backend, database, sometimes third-party services).

When to use: when you want to verify that a specific user flow works correctly across all the layers it touches. E2E tests are typically slower and more expensive to run than smoke or regression, so they're reserved for the highest-value paths.

Examples for B2B SaaS:

• New user signs up, verifies email, completes onboarding, creates first paid project
• Existing user upgrades from Free to Pro via Stripe checkout, sees plan reflect on the dashboard
• API customer hits a webhook endpoint, sees the run trigger and complete

Where they don't belong: as your wide-net regression check. E2E tests are deep but narrow. Running ten of them on every deploy gives you ten data points. Running fifty regression tests on every deploy gives you fifty.

The overlap, and why it confuses people

A single browser test can be all three at once. A "user logs in" test:

• Is a smoke test (if it runs in 10 seconds and only verifies the form posts and the dashboard loads)
• Is a regression test (if you wrote it because login broke once and you don't want it to break again)
• Is an E2E test (if it walks through the full session establishment, including the OAuth callback round-trip and a verification that the user's tenant data appears)

The classification depends on intent and depth, not on the technology. This is why people use the terms interchangeably and often correctly: a single Playwright file can serve all three purposes if scoped carefully.

What matters is that you know which job each test is doing, and that your test pyramid covers all three levels rather than collapsing them into one.

Quick decision matrix

If you can only do one

Most small SaaS teams can't run all three categories well. If you have to pick one to do properly, pick regression. It catches the most frequent class of customer-affecting bug (the backslide), runs in reasonable time, and doesn't require deep test engineering investment to get value from.

Smoke is easy enough to bolt on later (a one-line health check counts). E2E for the highest-stakes flow (checkout, signup) is worth the investment, but only after you have regression covering the basics.