CI Integration

Run your scaffolded TestCases in a pipeline, and control run mode, Self-Healing, and credentials without editing a file.

← Back to overview · 🇩🇪 Deutsch · ← Self-Healing Locators · Custom Steps →

---

The two commands every pipeline runs

Every command on this page runs from inside pm/ (see Quickstart — Step 5):

cd pm
npm ci
npx playwright test

npm ci (rather than npm install) is the standard CI choice — it installs exactly what pm/package-lock.json pins and fails instead of silently updating a dependency. npx playwright test resolves cc-testframework from pm/node_modules, the same way it does on your own machine; nothing CI-specific is needed for the CLI to work. Every npx cc-testframework <subcommand> call (set-run-mode, set-ai-key, self-healing status, …) resolves the same way.

---

Authenticating the install

npm ci/npm install reads pm/.npmrc, which references your license key rather than embedding it:

@meintest:registry=https://itsbusiness.vercel.app/api/tmgmt/npm/
//itsbusiness.vercel.app/api/tmgmt/npm/:_authToken=${CC_LICENSE_KEY}

Set CC_LICENSE_KEY as a secret in your CI provider — never as a plain workflow-file variable — and the install step authenticates the same way it does locally. See Quickstart — Step 3 for where the key comes from, and FAQ — Installation fails with 401 Unauthorized if the install step rejects it.

---

Environment variables you can set

All of these are read at the moment the relevant file loads (playwright.config.ts for the first two, pm/.npmrc for the third) — set them as CI secrets/variables, not as arguments on the command line, so they never show up in a log line or a process-list dump.

Variable	Controls	Default if unset
CC_LICENSE_KEY	Install authentication against the proxy registry (see above)	(required — install fails without it)
CC_RUN_MODE	failfast (abort on the first failed step) or failsafe (keep going past a failed assertion) — see Run and Debug	failfast
AI_API_KEY (or the legacy ANTHROPIC_API_KEY)	The BYOK key Self-Healing Vision uses to repair a broken locator — see Self-Healing — Cost and BYOK	(unset — Vision fallback is skipped, other Self-Healing behavior is unaffected)
SELF_HEALING_WRITEBACK	Whether a successful heal gets written back into your Control source file (Apply mode) — see Self-Healing — Setup	false — discovery only, nothing written
CC_PASSWORD_MANAGER_SECRET	Decrypts PasswordManager.json on a runner with no OS keychain — see Credential Management — Set up CI	(unset — reading a stored credential fails)
CC_AI_PRICE_JSON	Overrides the built-in per-model USD/token rate used in the heal-cost estimate — see Self-Healing — Seeing what a heal actually costs	Built-in example rate for the default vision model

💡 Leave SELF_HEALING_WRITEBACK unset (or false) in a normal test-run job. Apply mode opt-in exists so a CI run never mutates your repository unexpectedly — turn it on only for a dedicated job whose entire purpose is proposing locator fixes for review (a diff, a pull request), not for the job that gates your merge. See Self-Healing — How far Apply mode goes for the additional SELF_HEALING_WRITEBACK_MODE staging (working-tree / commit / commit-push) once you do turn it on.

Precedence: environment variable, always

Every one of these settings resolves through the same priority chain: an environment variable, when present, always wins over a persisted .cc-testframework.local.json file, which wins over the built-in default (see Self-Healing — Persisting the setup across sessions for where this is documented in full). For CI, prefer the environment variable over the npx cc-testframework set-run-mode <mode> / set-ai-key <key> setters that write to that file:

• The file is meant to be gitignored, per-machine, local-developer state — it doesn’t travel with a checkout, so setting it once wouldn’t survive into a fresh runner anyway.
• An environment variable set through your CI provider’s secrets store stays out of the checked-out source entirely — nothing to accidentally commit.
• A one-off override for a single job (e.g. CC_RUN_MODE=failsafe on just the nightly run) doesn’t require a separate step to persist and later revert a setting.

The set-run-mode/set-ai-key/config self-healing <action> CLI commands remain the right tool for a developer’s own machine — a one-time, per-machine setup that shouldn’t need repeating in every new shell. Use environment variables for CI, the setters for local dev; both feed the exact same resolution chain, so nothing behaves differently between the two.

---

CI-safe defaults already built into baseConfig

If your playwright.config.ts spreads the framework’s baseConfig (see API Reference — baseConfig), three settings already key off the standard process.env.CI flag that every major CI provider sets (GitHub Actions, GitLab CI, CircleCI, …) — no framework-specific variable needed:

Setting	Outside CI	Inside CI (process.env.CI truthy)
forbidOnly	false	true — a test.only(...) accidentally left in a commit fails the run instead of silently skipping the rest of the suite
retries	0	2 — a flaky step gets two more attempts before the test is reported failed
workers	Playwright’s own default (parallel, sized to available CPUs)	1 — tests run serially, avoiding resource contention on a typical CI runner (override with --workers=N if your runner has more headroom)

These three are the only CI-conditional settings in baseConfig — everything else (the ['list','html'] reporter pair, screenshot: 'on', video: 'on', trace: 'on-first-retry') behaves identically in and out of CI.

---

Headless by default — no virtual display needed

A project scaffolded from the shipped template doesn’t set headless at all, so a Web AUT run through npx playwright test uses Playwright’s own default (headless) unless you’ve deliberately switched a playwright.config.ts or a GlobalConfig.apps entry to headless: false to watch a test locally. A default scaffold therefore runs unmodified on a standard Linux CI runner — no Xvfb, no virtual display setup. If you did switch to headless: false for local debugging, either revert it before merging or make it environment-conditional — see FAQ — headless: false doesn’t show a browser on my Linux server for the pattern.

💡 Desktop (Appium/WinAppDriver) TestCases are a different story. They need a reachable Appium server with the target application installed on that same host — typically a Windows runner, not a standard Linux/macOS CI image. See FAQ — How does the framework know where to find the Appium server for Desktop tests? for the URL-resolution chain; setting up that Windows host itself is outside this page’s scope.

---

Machine-readable output for your pipeline

Two outputs are built for a script or a dashboard to parse, not just for a human reading the terminal:

• The list reporter’s inline lines — baseConfig’s reporter array always includes ['list'] alongside ['html'] (see API Reference — baseConfig), so every run prints a [PASS]/[FAIL] TS_x: ... line per step to stdout as it happens — the same lines you see locally, useful for a CI log viewer or any tool that tails the process output, without needing to open the HTML report.
• .self-healing-report.json — written inside pm/ by SelfHealingWritebackReporter whenever it’s registered (see Self-Healing — Setup), whether or not any heal actually occurred. Includes per-heal token usage and an estimated cost when Self-Healing Vision ran — see Self-Healing — The report file for the full schema. A CI job with Self-Healing enabled can read this file to post a summary comment or gate a review step.

---

A complete GitHub Actions example

name: e2e

on: [push, pull_request]

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - name: Install dependencies
        working-directory: pm
        env:
          CC_LICENSE_KEY: $
        run: npm ci
      - name: Run tests
        working-directory: pm
        env:
          CC_RUN_MODE: failfast
          AI_API_KEY: $
          CC_PASSWORD_MANAGER_SECRET: $
        run: npx playwright test
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: playwright-report
          path: pm/playwright-report/
          retention-days: 14

AI_API_KEY and CC_PASSWORD_MANAGER_SECRET are only needed if the job actually exercises Self-Healing Vision or a stored test-account credential — omit either one if your suite doesn’t use it. The upload-artifact step is optional but recommended: baseConfig always writes the HTML report to pm/playwright-report/ (see Run and Debug — Read the HTML report), and a CI runner’s local filesystem disappears with the job.

---

Any other CI provider

Nothing above is GitHub-Actions-specific. The two commands and the environment-variable table are the entire contract — a GitLab CI .gitlab-ci.yml job, a Jenkins pipeline stage, or a CircleCI config all reduce to the same shape:

export CC_LICENSE_KEY=<from your provider's secrets store>
export CC_RUN_MODE=failfast
export AI_API_KEY=<from your provider's secrets store>   # only if Self-Healing Vision runs
cd pm
npm ci
npx playwright test

---

Where to go next

• Run and Debug — CC_RUN_MODE, the HTML report, and reading a local failure
• Self-Healing Locators — Apply mode, writebackMode, and the report file’s full schema
• Credential Management — the full CC_PASSWORD_MANAGER_SECRET/OS-keychain priority chain
• FAQ — troubleshooting install, headless, and Appium-server-resolution issues

---

📧 Questions? Contact: jens.szelag@itsbusiness.ch

itsbusiness AG · Bern · Switzerland