Credential Management

Where the Anthropic API key and GitHub token live, and how to set them without touching your shell history.

← Back to overview · 🇩🇪 Deutsch · ← Step-Description Localization · FAQ →

What this is for

Two framework features call out to an external service on your behalf and need a credential to do it:

Both are stored the same way: in your operating system’s native credential store, never in a .env file, never in your shell history. This page covers the storage mechanism and the npx cc-testframework config command used to manage it. ANTHROPIC_API_KEY as a plain environment variable — the original setup path — still works exactly as before; everything below is an additional, more convenient option on top of it.

💡 Only using Self-Healing Vision? It also accepts a lighter-weight, provider-agnostic path that doesn’t need the OS credential store at all — the AI_API_KEY environment variable, or a key persisted per-machine with npx cc-testframework set-ai-key. See Self-Healing Locators — Cost and BYOK for the full precedence chain. The OS-credential-store setup on this page remains fully supported alongside it, and is still what the Custom-Step Authoring Agent (npx cc-testframework author) uses.

Where credentials are stored

Platform Store
Windows Windows Credential Manager
macOS macOS Keychain
Linux Secret Service (via your desktop session’s keyring daemon, e.g. GNOME Keyring or KWallet)

Every entry is stored under the service name cc-testframework, scoped to the OS user account running the framework. Nothing is transmitted anywhere except directly to the service the credential belongs to (Anthropic for the Vision key, GitHub for the token) — the framework itself never sees, proxies, or logs the raw value.

💡 No desktop keyring available (e.g. a headless Linux CI runner)? Reading and listing credentials degrade gracefully to “nothing found” rather than throwing — the framework falls back to the next tier in the priority chain below. Writing degrades to an error message pointing you at --value/--stdin or the plain environment-variable fallback instead.

The first-run prompt

The first time the framework needs a credential that isn’t configured anywhere yet, and it’s running in an interactive terminal, it asks for it once:

No anthropic-api-key found.
Enter your anthropic-api-key:
Save to system keychain for future runs? [Y/n]

The value you type is never echoed to the terminal. Answering Y (or just pressing Enter — that’s the default) stores it via the OS credential store described above, so every later run finds it automatically. Answering n still uses the value for the current run, just without persisting it.

This prompt only ever appears in an interactive terminal. In a non-interactive context (a CI job, a script piping input from elsewhere), the framework never prompts — prompting there would hang forever waiting for input that will never come. It reports the missing credential and exits instead; see Non-interactive setup below for the supported alternative.

Managing credentials via the config command 

npx cc-testframework config set <name> [options]      # store a credential
npx cc-testframework config get <name> [options]       # show a masked credential + its status
npx cc-testframework config delete <name> [options]    # remove a stored credential
npx cc-testframework config list [options]             # list everything currently stored
npx cc-testframework config types                       # list known credential types

npx cc-testframework config set anthropic-api-key
# Enter your anthropic-api-key: ****************
# Save to system keychain for future runs? [Y/n] y
# ✔ Stored 'anthropic-api-key' in system keychain (global)

npx cc-testframework config get anthropic-api-key
# sk-ant-A****...**xyz  (global, expires: 2026-08-01T00:00:00Z)

npx cc-testframework config list
# Stored credentials:
#   - anthropic-api-key    (global)
#   - github-token         (project: github.com/your-org/your-repo)

config get/config list never print the raw value — only a masked form (first 7 + last 3 characters, or **** for shorter values).

Options

Option Applies to Effect
--value <string> set Value to store, passed inline — visible in your shell history and process list, so prefer it for local, throwaway setup only.
--stdin set Reads the value from stdin instead — see Non-interactive setup.
(no flag) set Interactive prompt with hidden input — the default when running in a terminal.
--expires-at <ISO-8601> set Expiry timestamp, for an OAuth access token that has one. Omit it for a long-lived Personal Access Token or API key.
--source <name> set Informational tag, e.g. pat, oauth-device, oauth-app — shown back by get/list, purely for your own bookkeeping.
--project <id> set / get / delete Explicit project id, overriding git-remote auto-detection — see Project scoping.
--global set / get / delete Forces global scope for a normally project-scoped credential type.
--format text\|json get / list text (default, human-readable) or json (for scripts and other tooling that consume the output programmatically).
--help, -h all Prints usage.

Exit codes

Code Meaning
0 Success.
1 Not found, or the OS credential store is unavailable/failed.
2 User error — invalid arguments, unknown credential type, mutually exclusive flags, or a non-interactive set without --value/--stdin.
4 Credential-related error — the stored value is expired, has an invalid format, or the on-disk entry is corrupted. config get also returns 4 for an expired credential (it still prints the masked value and its status first).

Known credential types

Name Default scope Description
anthropic-api-key Global (one key per OS user) Anthropic API key for Vision-based Self-Healing and Custom-Step authoring (BYOK).
github-token Project (per-repository) A token for tooling that authenticates to GitHub — see the accepted formats below.

Run npx cc-testframework config types at any time to print this same list from your installed version.

Accepted GitHub token formats

Prefix Token kind Accepted
ghp_ Personal Access Token (classic) Yes
github_pat_ Fine-grained Personal Access Token Yes
ghs_ GitHub App server-to-server token Yes
gho_ OAuth App / OAuth Device-Flow access token Yes
ghu_ GitHub App user-to-server token Yes
ghr_ Refresh token No

A refresh token (ghr_) is never itself usable to call the GitHub API or push to a repository — it can only be exchanged for a new access token. Storing one under github-token would silently fail the first time it’s used, so config set rejects it upfront with a format error instead.

Project scoping

github-token defaults to project scope — a fine-grained PAT or OAuth token limited to a single repository is common, and different projects often need different tokens. The project id is auto-detected from git config --get remote.origin.url in your current working directory, normalized to <host>/<owner>/<repo> (so the SSH, HTTPS, and git:// forms of the same remote all resolve to the same stored entry).

# Auto-detects the project from the current repo's origin remote
npx cc-testframework config set github-token --stdin

# Explicit project id (e.g. running from a directory outside the target repo)
npx cc-testframework config set github-token --project github.com/your-org/your-repo --stdin

# One token for every project (a solo developer using a single PAT everywhere)
npx cc-testframework config set github-token --global --stdin

Reading (get/list, and the framework’s own internal resolution) checks the project-scoped entry for the current repository first, falling back to a global-scoped entry if no project-specific one exists. anthropic-api-key is always global — it doesn’t accept --project.

Priority chain for automatic resolution

Tools built on the framework (like the Authoring Agent CLI) resolve a credential automatically, checking each of the following in order and stopping at the first hit:

  1. An explicit value passed directly to the tool (e.g. --anthropic-api-key <value> on the author CLI).
  2. The corresponding environment variable (ANTHROPIC_API_KEY / GITHUB_TOKEN).
  3. The OS credential store, project-scoped entry.
  4. The OS credential store, global-scoped entry.
  5. The interactive first-run prompt — only in a terminal.

This means adopting the credential store is entirely optional: an existing export ANTHROPIC_API_KEY=... setup keeps working unchanged, and takes priority over anything stored via config set.

Non-interactive setup (for CI and automated tooling)

A script or an external tool driving the framework without a human at the keyboard should use --stdin rather than --value — the value never appears in a process list or shell history that way:

echo -n "$ANTHROPIC_API_KEY" | npx cc-testframework config set anthropic-api-key --stdin

The same pattern works for a token obtained through an external authentication flow (for example, a tool that walks a user through GitHub’s OAuth Device Flow on their behalf and then hands the resulting access token to the framework):

echo -n "$FRESH_ACCESS_TOKEN" | npx cc-testframework config set github-token \
  --stdin --source oauth-device --expires-at 2026-07-15T18:00:00Z

--expires-at and --source are both optional metadata — set them whenever the credential has a known expiry, so the framework can warn ahead of time and fail predictably once it actually expires (see below), instead of surfacing an opaque authentication error from the remote service.

💡 The framework never refreshes a token itself. It reads and stores whatever value it’s given — refreshing an expired OAuth token is the responsibility of whatever obtained it in the first place. When a token expires, request a new one through your provider’s flow and set it again with config set ... --stdin.

Expiry and the grace period

A credential stored with --expires-at carries that timestamp through every read. Three things can happen when the framework resolves it:

Situation Behavior
More than 5 minutes from expiry Used normally, no warning.
Within 5 minutes of expiry Used for this run, plus a warning that a refresh is coming due.
Already expired Treated as unusable — the framework falls through as if nothing were stored, and any caller relying on it fails fast with a credential-related error (see the Authoring Agent’s exit codes — code 4) rather than sending a doomed request to the remote service.

A credential stored without --expires-at (a long-lived Personal Access Token or API key, the common case) never triggers any of the above — it’s treated as valid indefinitely.

Progress events (for tooling built on top of the CLI)

Anything driving npx cc-testframework author programmatically (via onProgress / onCredentialProgress on runAuthorCli/runAuthorTestRunner — see API Reference) receives one of six credential-related event phases instead of having to scrape terminal output:

Phase Meaning
credential-expired A stored credential’s exp timestamp is in the past.
credential-expiring-soon A stored credential expires within 5 minutes but is still usable for this run.
credential-missing Nothing could be resolved anywhere in the priority chain.
credential-invalid A value was entered (interactively or via the API) that fails the credential type’s format check.
interactive-prompt-required The first-run prompt would normally appear, but the current context isn’t an interactive terminal.
interactive-prompt-shown The first-run prompt is being displayed right now.

Managing test-account credentials

This section covers a second, unrelated credential system: test-account credentials — the username/password pairs your TestCases use to log into the application under test, managed through Core.PasswordManager and stored in an encrypted PasswordManager.json file. It uses a different storage design than the Framework credentials described above, built around a two-layer model rather than the OS credential store alone.

💡 Prerequisite: the commands below assume the framework is already installed inside pm/ via the license-brokered proxy registry — see Quickstart — Step 3: Bootstrap authentication and scaffold your project if you haven’t set up CC_LICENSE_KEY yet.

Understand the two-layer model

  • The password-manager secret — a single AES-256 encryption key for your project. It lives in each developer’s OS keychain (the same native store described above) and is never written to the repository.
  • PasswordManager.json — the encrypted file itself. It lives inside pm/ (pm/2_Apps/1_Global/ by default) and is tracked by git like any other file, so it syncs to the team the normal way.
  • The project id — the keychain entry is scoped by name, derived from the name field in pm/package.json (falling back to the pm/ folder’s own name if that field is missing). This is why the secret is “for project X”, and why pm/package.json’s name field is a prerequisite for this to work reliably.

At runtime the framework combines the two: it looks up the password-manager secret for your machine, then uses it to decrypt the individual entry it needs from PasswordManager.json. Resolution checks, in order: the CC_PASSWORD_MANAGER_SECRET environment variable, then your OS keychain, then (deprecated) Core.Constant.passwordSecretKey in GlobalConfig.ts — see Migrate from the legacy passwordSecretKey below if you’re still using that. If none resolve, the command line tells you exactly what to run next.

Onboard the first team-member

The first developer on a project generates a fresh secret and stores it in their own keychain:

npx cc-testframework password init
# Existing team-secret? (paste hex) or generate new? (Enter):
# ✔ Password-manager secret initialized for project 'your-app' (stored in OS keychain).
#   Generated secret (share with your team via 1Password/Slack-DM, etc.): 3f9a...c1e0

Pass --secret <hex> or --stdin instead of the interactive prompt for a scripted setup. init refuses to overwrite an existing keychain entry unless you add --force.

Onboard subsequent team-members

Every other developer needs the same secret — the first team-member shares it out-of-band (a 1Password entry, a Slack DM, a secrets manager — anything other than committing it to git or attaching it to an email that ends up archived in a mailbox). Each colleague then runs:

npx cc-testframework password init --stdin
# paste the shared hex secret, press Enter

From that point on, their machine can decrypt every existing entry in PasswordManager.json and add new ones — no further onboarding step is needed per credential.

💡 Why out-of-band? PasswordManager.json travels with your repo the normal way (clone, pull, PR review). The secret deliberately does not — anyone who only has repo access, now or from a leaked backup, never gets the key needed to decrypt it.

Shared with other tools driving your test suite

PasswordManager.json and its resolution chain are not exclusive to this CLI — any other tool that manages or runs TestCases for the same project (for example, a companion test-management UI that scaffolds TestCases on a tester’s behalf) reads and writes the exact same file, through the exact same priority chain. There’s no separate credential vault to keep in sync between a CLI-driven developer and a UI-driven tester on the same team — the OS keychain fills the role for the CLI user that the companion tool’s own storage fills for its user, and both sides resolve the same decrypted password for the same username.

Add a credential 

npx cc-testframework password set --username admin --stdin
# paste the password, press Enter
# ✔ Stored credentials for 'admin'

Always use --stdin (or the interactive prompt, with no flag at all) rather than --value — a plaintext test-account password on the command line ends up in your shell history and in ps aux for the duration of the process. password get --username admin reads it back (masked by default, --raw to reveal it), password list shows stored usernames only, and password delete --username admin removes an entry.

Check the setup (password status)

password status reports what’s actually configured, without ever printing a secret or password value — useful when password get/password set fails with a “does not decrypt” error and you need to know why:

npx cc-testframework password status
# Password-manager status:
#   project:        your-app
#   secret source:  keychain
#   password file:  /path/to/your-app/pm/2_Apps/1_Global/PasswordManager.json
#   entries:        3
#   decryptable:    true

It prints the resolved project id (see Understand the two-layer model above), which of the three tiers the secret actually came from (env-var / keychain / the legacy field), the PasswordManager.json path, the entry count, and whether every entry currently decrypts with the resolved secret. --format json returns the same fields as JSON for scripting.

Rotate the secret (password rotate)

Re-encrypts every entry in PasswordManager.json with a brand-new secret and prints that new secret once, for you to share out-of-band the same way as during onboarding. Use it after a suspected leak (an old secret shared over an insecure channel, a machine that’s since been decommissioned) or as a periodic hygiene practice:

npx cc-testframework password rotate
# ✔ Rotated the password-manager secret and re-encrypted 3 entries.
#   New secret (share with your team via 1Password/Slack-DM, etc.): 7c1f...9ae2

After rotating, every other team-member re-runs npx cc-testframework password init --stdin --force with the new secret — their old keychain entry no longer decrypts the file, exactly like a first-time onboarding.

Read a credential in your test

Read the stored password through the shipped TS_GetPassword step instead of calling a Core service directly from your TestCase — it takes only its one business parameter, no leading refId:

const pw = await Project.Utilities.TS_GetPassword('admin');
await Project.<YourApp>.TS_Main_Passwordfield_Fill(pageLogName, sectionName, 'Password', pw);

TS_GetPassword returns the decrypted password and logs only the username to the report — the password value itself is never written anywhere in the output. The lower-level Core.PasswordManager.getPassword(...) API this step wraps still exists for advanced/programmatic use outside a TestStep (see API Reference — Section 13); inside a TestCase, TS_GetPassword is the recommended path. See Migrating to v0.22.0 below if you’re updating from an older call shape.

Set up CI

CI runners have no OS keychain, so set the secret through an environment variable instead — CC_PASSWORD_MANAGER_SECRET takes priority over the keychain and the legacy field, and is read on every run without any password init step:

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - name: Install dependencies
        working-directory: pm
        env:
          CC_LICENSE_KEY: $
        run: npm ci
      - name: Run tests
        working-directory: pm
        env:
          CC_PASSWORD_MANAGER_SECRET: $
        run: npx playwright test

npm ci reads pm/package-lock.json and pm/.npmrc — the latter already carries the ${CC_LICENSE_KEY} reference scaffolded in Quickstart — Step 3, so setting the same-named secret in your CI provider is enough. Store both values in your CI provider’s own secrets store (GitHub Actions secrets, Vault, Azure Key Vault, …) — never as a plain workflow-file variable. PasswordManager.json itself is already checked into the repo, so no extra step is needed to get it onto the runner.

Override the secret for a single run

To try a different secret without touching the keychain — for example, testing what happens with a teammate’s secret before committing to password init --force — pass it inline for one invocation, from inside pm/:

CC_PASSWORD_MANAGER_SECRET=<value> npx playwright test 3_Cases/TC_Login.spec.ts

This uses tier 1 of the priority chain above (Understand the two-layer model) for that single run only — it wins over the keychain and the legacy field, and nothing is persisted anywhere once the process exits.

Migrating to v0.22.0

Breaking change. Before v0.22.0, TS_GetPassword took a leading refId (TS_GetPassword(refId, username)), matching the call shape of a TS_Main_* step written against your own App. As of v0.22.0 it drops that argument — the same “no screen element, no reference screenshot, nothing to bind” reasoning covered in the OS App and FTP App migration notes applies here too.

// before (≤ 0.21.0)
const pw = await Project.Utilities.TS_GetPassword('', 'admin');

// after (0.22.0)
const pw = await Project.Utilities.TS_GetPassword('admin');

Update every TS_GetPassword call site in your project by removing the leading empty-string argument.

Migrate from the legacy passwordSecretKey

  1. Find your project’s current value of Core.Constant.passwordSecretKey in GlobalConfig.ts.
  2. Move it into the keychain: npx cc-testframework password init --secret <the-existing-key>.
  3. Delete the Core.Constant.passwordSecretKey = '...' line from GlobalConfig.ts and commit.

PasswordManager.json doesn’t need to change — the encryption format is identical either way, only where the key comes from changes. Until you migrate, the legacy field keeps working, with a one-time-per-run deprecation warning pointing back at this section.

Threat model

This design protects against a leaked or briefly-public repository, a backup or mirror exfiltration, and anyone mining git log/blame history — in every one of those cases, only the encrypted JSON is exposed, never the key needed to read it. It does not protect against a developer machine that’s already compromised (an attacker who can read your keychain can decrypt exactly what you can), someone shoulder-surfing the secret during onboarding, or a leak of CC_PASSWORD_MANAGER_SECRET from your CI provider’s own secrets store — securing that value is your CI provider’s responsibility, the same as any other CI secret.

Where to go next

📧 Questions? Contact: jens.szelag@itsbusiness.ch

itsbusiness AG · Bern · Switzerland