How to automate regression testing with Claude Code

Give Claude Code a browser tool and it can open the page it just changed, click through it and tell you it works. That check is real, and it ends with the session. Next week another change touches the same component, and the check runs again only if someone remembers to ask.

This guide gives those checks a place to live. Regresco runs an MCP server at https://regresco.com/mcp. Connected to it, Claude Code saves regression flows in your Regresco project, runs them in real Chromium on Regresco's cloud workers and reads what failed. The flows stay in your account after the chat ends, and they run again from the dashboard, on a schedule or from CI. I build Regresco, so read the rest with that in mind.

Why agent-written code needs tests that outlast the session

The agent checks what it changed. Ask for a new billing page and it tests the billing page. If it edited a shared layout or an auth helper on the way, login can break without anyone opening login.

The check lives in the conversation. A browser session the agent drove last Tuesday leaves a chat log, and nothing in your project runs it again.

Building gets faster and checking doesn't. When a feature takes minutes, clicking through signup, login and checkout before each release becomes the slowest step, and the slowest step is the one that gets skipped.

A saved flow runs the same steps every time and can run with nobody asking. It also keeps a history, so a failure can be compared with the last time the same step passed. Regression testing for B2B SaaS covers which paths to protect first.

What “MCP regression testing” means here

Search for MCP regression testing and many results are about testing MCP servers themselves. This guide is about the other meaning: a coding agent uses an MCP server to save and run regression tests for your web app.

Two kinds of server get mixed up here. One drives a browser for the agent right now. Playwright MCP is one of these: the agent reads the page through accessibility snapshots and clicks through it during the conversation. The other kind stores tests that keep running. Regresco is that kind, with 15 tools to list, create and edit projects and flows, start runs and read results. None of them deletes anything.

Ask each option what still runs next week:

The two kinds work well together: a browser tool to look at a page and find selectors, Regresco for the checks that should run before every release. The same Regresco server works from Claude Code, Claude and ChatGPT, and the MCP guide also has setup steps for Cursor and Codex. The rest of this guide uses Claude Code.

What you need

• A Regresco account. MCP is on every plan, including Free, which has 20 runs a month and one project.
• A staging or preview URL that the internet can reach. Localhost and private networks are blocked, and a site behind Cloudflare or a CAPTCHA needs an extra step.
• A test account on staging for flows that sign in. Never a real one.
• Claude Code.

Connect Claude Code

On your own machine, sign in with OAuth. For CI, or a machine where you can't sign in, use a personal token.

Sign in with OAuth

Add the server:

claude mcp add --transport http --scope user regresco https://regresco.com/mcp

--scope user makes Regresco available in all your projects. Then run /mcp in Claude Code, choose regresco and Authenticate. Regresco's consent screen opens in your browser. Reading is always included. To let Claude Code save flows and start runs, tick Create and edit test flows and Start test runs, and Create and edit projects if it should create projects too. Then select Allow access.

Or use a token

Create a token in Settings → AI agents (MCP). It starts with rgr_mcp_, is shown once, and can only read until you give it permissions when you create it. Keep it in an environment variable, so it never appears in a command, your shell history or Claude Code's configuration.

On macOS or Linux, paste the token at the prompt (it isn't shown while you type), then start Claude Code from the same terminal:

printf 'Regresco MCP token: '; read -rs REGRESCO_MCP_TOKEN; echo
export REGRESCO_MCP_TOKEN

On Windows PowerShell, this saves it for your user account. Open a new terminal afterwards:

$t = Read-Host "Regresco MCP token" -AsSecureString
[Environment]::SetEnvironmentVariable("REGRESCO_MCP_TOKEN", [Net.NetworkCredential]::new("", $t).Password, "User")
Remove-Variable t

Then add Regresco with a header that names the variable:

claude mcp add --transport http --scope user regresco https://regresco.com/mcp --header 'Authorization: Bearer ${REGRESCO_MCP_TOKEN}'

The single quotes stop your shell from replacing the variable. Claude Code saves the text ${REGRESCO_MCP_TOKEN} and fills in the token each time it connects.

Check the connection

claude mcp list should show:

regresco: https://regresco.com/mcp (HTTP) - ✔ Connected

In a session, /mcp lists the Regresco tools. Ask Claude Code to list your Regresco projects, and it should name the ones you see in the dashboard.

To share the setup with your team without sharing a token, commit a .mcp.json at the root of the repository. Claude Code expands ${VAR} in that file, so every developer keeps their own token in their own environment:

{
  "mcpServers": {
    "regresco": {
      "type": "http",
      "url": "https://regresco.com/mcp",
      "headers": { "Authorization": "Bearer ${REGRESCO_MCP_TOKEN}" }
    }
  }
}

A working session, step by step

Here is how a session goes on a staging app with a login. The quoted lines are what you type, and the names in code are the tools Claude Code calls.

1. Find or create the project

“List my Regresco projects.” (list_projects) If none of them points at this app: “Create a Regresco project for https://staging.example.com.” (create_project) The Free plan has one project, so reusing it matters.

2. Save a flow

“Add a regression flow for login: open /login, sign in with the staging test user, check that the dashboard heading appears.” (create_flow)

Regresco's tools don't read pages, so the selectors have to come from you, from your code or from a browser tool of Claude Code's own. In a repository, Claude Code can read the component that renders the form and take the field names and button labels from there. It also shouldn't ask for the password in the chat. Regresco's server tells it to save a placeholder and ask you to type the real test password into the step in the dashboard.

What it sends looks like this, trimmed from the create_flow example in the MCP guide. The password value is a placeholder:

{
  "name": "Login with email",
  "description": "Signs in with the staging test user and checks the dashboard",
  "steps": [
    {
      "action": "navigate",
      "url": "https://staging.acme.dev/login"
    },
    {
      "action": "fill",
      "selector": "input[name='email']",
      "value": "qa@acme.dev"
    },
    {
      "action": "fill",
      "selector": "input[type='password']",
      "value": "staging-test-password"
    },
    {
      "action": "click",
      "selector": "button[type='submit']"
    },
    {
      "action": "assert",
      "selector": "h1:has-text('Dashboard')"
    }
  ]
}

Read what the agent saved before you trust it. An assert step checks that an element is visible, not what it says, so the text that matters goes into the selector, as in h1:has-text('Dashboard'). A flow that ends on a click proves the button exists and nothing more.

3. Run it

“Run the login flow.” (run_tests) While you iterate on one flow, Claude Code passes its flowId, which runs only that flow. It also sends an idempotencyKey, so a retried call within 24 hours returns the same run instead of starting a second one:

{
  "projectId": "cmwamxl5x0003bz9oof5xg2gv",
  "flowId": "cmwamxl6m0005bz9oah0k6w1m",
  "idempotencyKey": "login-fix-attempt-1"
}

run_tests returns straight away with the run QUEUED. Claude Code then calls get_run about every 30 seconds until the run is finished. Most runs finish within a few minutes. Each call that starts a run uses one of your plan's runs, and reading results uses none.

4. Read what failed

“Show what failed.” (get_run_failures, then get_run_artifacts for screenshot links) For each failed step, the agent gets the facts. This is part of the example in the guide:

{
  "action": "assert",
  "selector": "h1:has-text('Dashboard')",
  "classification": "REGRESSION",
  "error": "The dashboard heading did not appear after sign-in (waited 5000ms).\nExpected: visible\nReceived: hidden",
  "consoleErrors": [
    {
      "level": "error",
      "text": "POST https://staging.acme.dev/api/session 500 (Internal Server Error)"
    }
  ],
  "autoHealed": false,
  "inBaseline": "passed"
}

In that example the step passed in the earlier run, and the page's own sign-in request returned a 500. The facts point at the application rather than the test, so Claude Code can fix the endpoint and run the same flow again. When the facts don't support a cause, it shouldn't invent one. Every failure response carries a note saying so, and Regresco marks its own selector hints and explanations as unverified. How Regresco classifies failures explains the three labels.

5. Run everything before you merge

“Run every flow in the project and summarize the failures.” (run_tests without a flowId, then get_run and get_run_failures) A test passed only when get_run reports PASSED. If it counts healed steps, a click or fill passed only after Regresco used another element with exactly the same name. That deserves a look, and the run page names both selectors and offers Use this selector. Assert steps are never replaced. When a healer hides a regression shows why that rule matters.

Make it part of how Claude Code works

You can ask for all of this in every session, but it's easier to write it down once. Claude Code loads a project's CLAUDE.md at the start of every session, so a few lines there turn the runs into a habit.

Illustrative, not run:

## Regression checks (Regresco)
- The Regresco project "Staging" holds the browser regression flows for this app.
- After a change that touches the UI, run the affected flow with run_tests and its flowId, and poll get_run.
- Before you say a task is done, run the whole project and report what get_run_failures returns.
- When you add a user-facing feature, add a flow for it with create_flow. Take selectors from the code.
- Never put real credentials in a step. Save a placeholder and tell me to fill it in the dashboard.

Claude Code treats CLAUDE.md as context rather than enforced configuration, so on a busy day it can skip a line. That's the reason to let the same flows run without the agent too: set a daily or weekly schedule on the project page, and trigger the project from your pipeline after each deploy to staging. Regression checks in CI/CD has the webhook call and the GitHub Actions and GitLab CI steps.

What it looked like on regresco.com

On 3 November I had Claude Code cover regresco.com through this same server, as 24 flows with 706 steps. Thirteen flows walk the public site, including every address in its sitemap, and eleven walk the logged-in dashboard. A run of the whole project passed. The project is still in the account, and I run it again after changes to the site.

Limits worth knowing first

• Regresco runs Chromium only. A flow uses a desktop screen (1280x720) or a phone's size and touch (412x839) emulated in Chromium. That isn't a real phone, and it isn't Safari.
• A flow has ten step types and runs no custom JavaScript. It can't double-click or mock the network, and it can't check that an element is gone or what a field holds. Tests that need those belong in Playwright in your repository; Playwright Test Agents and Regresco shows what carries over when you import them.
• run_tests refuses a new run while 3 runs are queued or running for your account, wherever they were started.
• A flow saved through MCP can have up to 100 steps.
• Screenshot links expire after 15 minutes. Playwright traces stay in the dashboard, because they record typed values and request headers in full.
• Every started run counts against your plan: 20 a month on Free, 200 on Pro, 1,000 on Business (pricing).

Questions