Registry stack documentation: machine-readable Markdown.
Index of all pages: https://docs.registrystack.org/preview/llms.txt
Full corpus: https://docs.registrystack.org/preview/llms-full.txt

# Rotate credentials, keys, certificates, and trust

> Rotate Registry Relay and Registry Notary trust material while preserving product ownership, overlap, recovery, and staged traffic admission.

Use this procedure to rotate a source credential, caller key, Notary-to-Relay workload token,
configuration signer, credential-issuance signer, certificate, or trust anchor without exposing
secret material or widening authority.

## Prerequisites

- Identify the material, every consumer, its current secret or trust reference, and its expiry.
- Preserve a verified recovery set for the current Relay and Notary product configurations.
- Keep caller traffic outside the staged instances until the affected product checks pass.
- Use synthetic or institution-approved canaries. Do not use personal data for a rotation probe.

## Ownership and trust boundary

Registry Relay owns source destinations, source credentials, private certification authority
material, mutual TLS keys, source protocol credentials, and Relay caller keys.
Registry Notary owns its Relay workload token, Notary caller policy, and evidence-signing keys.
The deployment operator owns secret storage, certificates, product trust anchors, traffic
admission, and revocation.

Configuration signing keys and trust anchors are product-specific.
Relay and Notary bundles have separate signatures, anchors, anti-rollback state, and activation
events.
A combined project has no signed project-root bundle or atomic activation coordinator.

{/* Evidence: products/notary/docs/configuration-trust-and-integrity.md,
    products/platform/docs/governed-configuration.md,
    crates/registry-relay/docs/ops.md, and
    products/notary/docs/signing-key-provider.md. */}

## Classify the rotation

1. Treat suspected exposure as an incident.
   Revoke or disable the affected material, stop affected traffic, and preserve redacted audit
   evidence before routine rollout work.
2. Decide whether consumers need an overlap window.
   Caller keys, certificate chains, configuration signers, and public credential-verification
   keys normally need overlap.
3. Decide whether the change remains in the secret plane or changes a governed reference.
   A secret-plane change retains the reviewed reference and replaces the value through the
   secret provider.
   A governed change produces a reviewed product configuration and a new signed product bundle.

The [Notary deployment hardening runbook](../../../products/registry-notary/deployment-hardening-runbook/)
defines the incident boundary.
The [Relay operations runbook](../../../products/registry-relay/ops/) defines Relay API-key and
runtime-secret rotation.

## Rotate source credentials, certificates, or source trust

1. Add the replacement to the operator-owned Relay secret and trust boundary.
   Use a versioned reference when the provider treats references as immutable.
2. Change only the affected Relay environment binding.
   Do not change the source origin, method, path rules, projection, or limits as part of a
   credential rotation.
3. Run the offline environment gate:

   ```sh
   registryctl preflight \
     --project-dir <project> \
     --environment <environment> \
     --format json
   ```

4. Compare the change with the verified Relay baseline, then build, sign, and verify the Relay
   product input.
   Use the [registryctl CLI reference](../../../reference/registryctl/) for the exact baseline and
   bundle commands.
5. Stage Relay without caller traffic.
   Require health, readiness, audit, and redacted deployment posture to pass.
6. Run one authorized, bounded source canary when the source authority permits that test.
   Offline preflight does not contact the source.
7. Move callers to the staged Relay and retire the old secret or trust material after the overlap
   window closes.

Private CA and mutual TLS changes stay in Relay.
Do not copy source certificates, keys, or destinations into a Notary bundle.

## Rotate caller keys and the Relay workload token

For a Relay API key, generate a new raw key and fingerprint:

```sh
registry-relay generate-api-key --id caller-rotation-2026-01
```

Store the emitted raw key only in the authorized client secret store.
Store the fingerprint under the configured Relay secret reference, keep both caller entries during
the overlap window, restart or roll Relay, move the caller, then remove the old entry.
Relay does not provide live keyring reload in the documented v1 path.

For the Notary-to-Relay workload token, update the Relay workload policy and atomically replace the
owner-readable token file used by Notary.
Stage Relay first, then stage Notary against Relay and require the complete consultation contract
check.
Admit caller traffic only after both products are ready.

## Rotate configuration signers and trust anchors

1. Create or inspect product-specific trust-anchor operations with:

   ```sh
   registryctl anchor --help
   ```

2. Add the replacement public signer to the affected product's local trust anchor before a bundle
   depends on that signer.
3. Distribute the updated anchor through the deployment trust path and restart staged instances.
4. Sign the next product bundle with the replacement key and a strictly higher sequence.
5. Run the stateless bundle check against the affected product anchor:

   ```sh
   SIGNED_PRODUCT_BUNDLE=operator-inputs/signed-relay-bundle
   PRODUCT_TRUST_ANCHOR=operator-inputs/relay-trust-anchor.json
   registryctl bundle verify \
     --bundle-dir "$SIGNED_PRODUCT_BUNDLE" \
     --anchor-path "$PRODUCT_TRUST_ANCHOR"
   ```

   This command verifies the signature, product binding, trust anchor, and signed file closure.
   It does not read product anti-rollback state or establish local rollback eligibility.
6. For an existing Relay state lineage, validate the later candidate or restart read-only against
   that node's accepted state:

   ```sh
   SIGNED_RELAY_BUNDLE=operator-inputs/signed-relay-bundle
   RELAY_TRUST_ANCHOR=operator-inputs/relay-trust-anchor.json
   RELAY_ROLLBACK_STATE=operator-state/relay-config-state.json
   registry-relay config verify-bundle \
     --bundle-dir "$SIGNED_RELAY_BUNDLE" \
     --anchor-path "$RELAY_TRUST_ANCHOR" \
     --state-path "$RELAY_ROLLBACK_STATE"
   ```

7. For an existing Notary state lineage, run the equivalent Notary-owned check:

   ```sh
   SIGNED_NOTARY_BUNDLE=operator-inputs/signed-notary-bundle
   NOTARY_TRUST_ANCHOR=operator-inputs/notary-trust-anchor.json
   NOTARY_ROLLBACK_STATE=operator-state/notary-config-state.json
   registry-notary config verify-bundle \
     --bundle-dir "$SIGNED_NOTARY_BUNDLE" \
     --anchor-path "$NOTARY_TRUST_ANCHOR" \
     --state-path "$NOTARY_ROLLBACK_STATE"
   ```

   Each product command verifies the candidate against that product's existing local state without
   persisting an acceptance.
   These read-only commands require accepted state to exist and are for later candidate validation
   and restarts.
   A genuinely absent, version-specific state path is initialized only by the documented product
   boot path, not by either read-only verifier.
8. Remove or disable the old signer only after every retained deployment and rollback set has an
   accepted recovery path.

Repeat this workflow independently for Relay and Notary when both signers change.
Do not interpret two successful verifications as atomic project activation.

## Rotate Notary evidence-signing keys

Add a new key with a new `kid` and `status: active`.
Move the intended credential or federation profiles to the new named key.
Change the old key to `publish_only` for the full verifier and credential lifetime window, then
disable or remove the old key through its governed cleanup path.
Never reuse a `kid` for different key material.

The [Notary signing key provider guide](../../../products/registry-notary/signing-key-provider/)
defines local JWK and PKCS#11 provider configuration, readiness behavior, and the complete key
lifecycle.

Audit hash-secret rotation is a separate evidence-lifecycle event.
Retain the old secret under the audit retention policy because changing the secret breaks lookup
correlation with older pseudonyms.

## Expected evidence

Retain:

- The redacted preflight and semantic-comparison reports.
- The verified product bundle report and product-specific sequence.
- Health, readiness, audit-write, and redacted posture results from the staged instance.
- A synthetic or authorized bounded canary result.
- The old-material retirement time and the approved overlap window.

Evidence records may contain key ids, product ids, scopes, and certificate metadata.
They must not contain raw keys, fingerprints, private certificates, tokens, environment values, or
full configuration dumps.

## What this proves

The offline gates prove that required references exist, file posture is acceptable, the
environment does not widen the authored contract, and the generated product input validates.
`registryctl bundle verify` proves the stateless signature, binding, trust-anchor, and file-closure
checks.
The product-owned `config verify-bundle` command additionally proves read-only local rollback
eligibility against that product's existing accepted state.
Neither command persists bundle acceptance.
Staged readiness and a bounded canary prove the tested runtime path.

These gates do not prove country approval, legal authority, every source operation, every caller
migration, remote audit retention, or live interoperability that was not tested.

## Roll back or recover

Before the replacement serves traffic, restore the prior secret reference, anchor, bundle, and
runtime recovery set.
After a higher configuration sequence has been accepted, do not delete anti-rollback state or
activate an older sequence.
Issue a reviewed higher-sequence recovery bundle, or use the exact documented, expiring local
break-glass process.

Keep old Notary public verification keys published while credentials signed by those keys can
still be verified.
Keep both products staged and traffic blocked when Relay and Notary no longer agree on the
consultation contract.

## Escalate

Escalate to the product security owner when material may be exposed, a private key or token reaches
logs, trust cannot overlap safely, a certificate changes source identity, an old Notary public key
cannot remain published, anti-rollback state is missing, or a canary requires live country data.

## Next

- [Compare a baseline and reapprove a source change](../compare-and-reapprove-source-change/)
- [Inspect and diagnose a running deployment](../inspect-and-diagnose/)
- [Back up and restore state](../../backup-and-restore/)