Retention and persistent state

Use this page to decide which Registry Stack state needs backup, off-host shipping, expiry, or deletion policy outside the products. It covers the shared audit writer and the retained state of Registry Relay, Evidence Gateway, Base Registry Engine, and Registry Render, with audit upgrade steps for Casework and Scheduling. Product retention commands remove only the state each product owns; your institution supplies the retention policy.

Operator boundary

Registry Stack gives operators rotation knobs, TTL-bound protocol stores, and pseudonymized audit handles. It does not decide the legal retention period for a deployment, delete source-registry records, or prove that a local rotating file kept every historical audit event.

Two consequences follow from that boundary:

• Treat local audit files as neither tamper-evident nor complete: they are not chained, and retention deletes sealed files by age. Ship them to append-only storage when integrity or completeness matters.
• Treat keyed pseudonyms as a linkability control, not as erasure. Destroying or rotating the HMAC secret prevents recomputing old pseudonyms from raw values, but it does not delete audit records, cache entries, or source data.

Durable state and externally retained records

Store	What it can contain	Expiry or rotation	Operator control
Relay audit file	Attempt request entries and refusal or terminal response entries binding the registry, resource, operation, access profile, disclosure profile, selected-field identifiers, contract revision, and source revision. It excludes tokens, selectors, raw principals, source values, response values, and raw identifiers.	Relay writes through the shared audit writer Evidence Gateway uses. The active file rotates to <audit.path>.<sequence> when an append would pass audit.rotateBytes (100 MiB by default), and sealed files last modified more than audit.retainDays ago (90 by default) are deleted when the writer opens or rotates. A stdout destination leaves rotation and retention to the log collector.	Configure audit.destination, audit.path, and optionally audit.rotateBytes and audit.retainDays in runtime.yaml. Ship sealed files off host before retention deletes them; the mechanics are identical to Ship Evidence Gateway audit records off host, since both products share the same writer and file naming.
Evidence Gateway audit file	A JSONL log of the access-attempt entry before every source read and the release, denial, or failure entry before every response. Audit carries reviewed identifiers and decision categories, never raw selector values, source values, credentials, tokens, or raw subject identifiers.	audit.rotateBytes (100 MiB by default) is a per-file rotation threshold, not a total ceiling. When an append would pass it, the runtime renames the active file to <audit.path>.<sequence> and opens a new active file online, with no operator action. Sealed files last modified more than audit.retainDays ago (90 by default) are deleted when the writer opens or rotates; the active file is never deleted.	Set audit.destination, audit.path, and optionally audit.rotateBytes and audit.retainDays in runtime.yaml; the pseudonym key is the bundle’s audit.hashKeyRef. Ship sealed files to append-only storage before retention deletes them, and never touch the active file while the service runs.
Base Registry Engine PostgreSQL database	The registry’s own database. registry_data holds the current row of every record, registry_internal holds the retained revisions, history-erasure coverage, the webhook outbox, the change-request proposal and target snapshots, and the cached idempotency results, and registry_source, registry_derived, and registry_context hold views and functions the compiler generates. Records, their revisions, and proposal snapshots carry whatever the project’s entities declare, personal data included.	Only a retained webhook payload expires on a schedule: eventDelivery.payloadRetentionDays in runtime.yaml defaults to 7 days and is capped at 30, and the runtime clears a payload on the first successful delivery or once its expiry has passed. Revisions and proposal snapshots are kept until an operator command removes them.	Back up the database; Base Registry Engine writes no local copy of it. bregctl history erase and request-retention erase remove retained history and proposal detail; audit files need separate shipping and backup. Retain, erase, and audit states what each one destroys and what it leaves.
Base Registry Engine audit files	Minimized request, response, refusal, and maintenance entries with keyed references, separate from the database. Runtime instances and operator companion commands have separate files.	The shared writer rotates at audit.rotateBytes (100 MiB by default) and deletes sealed files older than audit.retainDays (90 days by default) on open or rotation.	Give every process its own absolute path, and ship runtime and bregctl companion files before expiry. With destination: stdout, the collector owns durability, rotation, and retention, and bregctl companion commands write their entries to stderr so their own report keeps stdout; collect both streams.
Render audit file	A request entry accepted before a render’s worker starts and a response entry accepted before the response leaves, both sharing one correlation, a random id the server draws for every call. The caller’s Idempotency-Key is recorded only as the record’s correlationId, since two calls may carry the same key. Entries carry hashes, versions, the renderer and Typst pin, caller fingerprint, and trace/correlation ids, never record data values or asset bytes.	Render writes through the shared writer Evidence Gateway and Relay use. The active file rotates to <audit.path>.<sequence> when an append would pass audit.rotateBytes (100 MiB by default), and sealed files last modified more than audit.retainDays ago (90 by default) are deleted when the writer opens or rotates. A stdout destination leaves rotation and retention to the log collector.	Configure audit.destination, audit.path, and optionally audit.rotateBytes and audit.retainDays in the runtime file. Ship sealed files off host before retention deletes them; the mechanics are identical to Ship Evidence Gateway audit records off host, since all three products share the same writer and file naming.
Operator-owned config, source, and secret paths	Relay’s runtime.yaml, the sealed package at package.root, the secret files under secretProviders.file.root, and the bound SQLite source path; Evidence Gateway’s governed bundle and runtime document; Base Registry Engine’s package root, runtime file, project sources, module locks, generated files, test receipt, and the bregctl data checkpoint, sidecar, and export output files on the operator host; secret references for all three. Source files and a bregctl data export output can contain personal data.	Registry Stack does not expire these files, except through the specific audit and cache mechanics listed for each store.	Mount source data read-only where possible; back up config, packages, and secrets through your platform controls.

Relay V2 keeps no other durable, product-owned state beyond the audit file and the operator-owned paths in Durable state and externally retained records: no shipper-acknowledgement cursor file, no server-side ingest cache, no configuration-signing or anti-rollback state, and no separate consultation database. Relay V1 had all four; the Relay recovery state section explains why none of them carry forward.

Process-local caches and client-held state

Store	What it can contain	Expiry or rotation	Operator control
Relay OIDC JWKS cache	Issuer signing keys and negative lookup entries, not subject records. Applies only when authentication.oidc is configured; a package where every access profile is public can run without it and with no cache at all.	In process only. Relay V2 exposes no cache-lifetime override, so the shared platform defaults apply: 600 second positive cache TTL and 60 second negative cache TTL.	Restart clears the cache.
Evidence Gateway OIDC JWKS cache	Issuer signing keys and negative lookup entries for the configured authentication.oidc.jwksSource.uri, not subject records.	In process only. Evidence Gateway exposes no product-level override, so the shared platform defaults apply: 600 second positive cache TTL and 60 second negative cache TTL.	Restart clears the cache. An unreachable key set is retried and reported at a bounded interval rather than silently ignored.
Relay response headers	No stored response cache. Relay validates certain public snapshot responses with a strong ETag and answers a matching conditional request with 304 Not Modified; every other response, including metadata, defaults to Cache-Control: no-store.	Not time-bound; validators change when the underlying package, source revision, or requested representation changes.	Not configurable. Behavior follows the source profile and access profile compiled into the package.
Relay pagination cursors	Client-held, authenticated, and encrypted cursor payloads binding the source and contract revisions, operation, access profile, disclosure profile, filters, order, selected fields, and authorization context. No plaintext filter, order, or bbox value.	Not stored server-side. Each cursor carries an expiry checked against cursor.maximumAgeSeconds, which the runtime defaults to 300 seconds only when the whole cursor section is absent, and becomes invalid immediately if the bound request context no longer matches. Cursors apply to snapshot-source list and search operations alike; live read-only sources do not support pagination.	Configure cursor.integrityKeyRef and, optionally, cursor.maximumAgeSeconds in runtime.yaml. Treat cursor tokens as opaque, client-held request context.

Relay V2 has no auth-failure throttle and no source-side OAuth token cache: caller authentication is a stateless bearer-token check against the configured OIDC issuer, and a source is a read-only SQLite file with no credential of its own to cache a token for.

Audit retention

Relay and Render write JSONL audit entries through the same shared writer Evidence Gateway uses, rotating the active file at audit.rotateBytes and deleting sealed files older than audit.retainDays.

Entries are not chained, so the local file detects neither edits nor deletion. It does not prove that older rotated-away records, or earlier retained records, still exist. Off-host durability and tamper evidence are entirely the operator’s responsibility: Relay V2 has no deployment-posture report, no evidence_grade startup mode, and no doctor command, so it never surfaces a finding about local-only retention risk the way Relay V1 did. There is also no shipper-acknowledgement cursor to check a shipper’s progress against. Treat a running deployment’s /ready result as evidence the audit writer was healthy at that moment, not as proof of complete off-host retention.

Never truncate, rewrite, or reserialize the active audit file as part of cleanup: the writer treats any external change to it as a fault and stops, and audited requests fail closed until a restart. Copy sealed files to append-only storage before audit.retainDays deletes them.

A crash, a killed process, or a full disk can leave the active file’s final line incomplete (any byte other than a newline at end of file). The next start recovers it without operator action: the writer copies the bytes after the last complete line to the owner-only (0600) side file <audit.path>.torn, syncs it, truncates the active file to its last complete line, and logs the side file’s path and byte count, never its bytes, at error level. An entry is acknowledged only after the write holding it is synced, so the torn bytes belong to no acknowledged entry and no acknowledged entry is lost. doctor and companion checks accept a destination the writer would recover this way and change nothing.

Archive the side file with the sealed files, then remove it. The writer never overwrites it: if a later torn line finds <audit.path>.torn holding other bytes, the start is refused until that file is moved. A tail longer than any entry, or a file whose first entry is in another format, is still refused unchanged; archive that file and restart with a fresh audit.path. Do not edit the active file to repair it; editing it while a writer holds it is the same external-change fault described above.

Evidence Gateway retention

Evidence Gateway has no application database and persists no selector, source, evidence, or response data. Its durable state is its governed signer configuration and public keys plus the audit file described in Durable state and externally retained records. The external Transit service retains the production private key. An external durable audit service may own its own storage, but Evidence Gateway itself does not maintain one.

The runtime recognizes a sealed file by its <audit.path>.<sequence> name, with an eight-digit sequence, and never deletes the active file. Retention deletes sealed files in sequence order, oldest first, and stops at the first one whose last modification is inside audit.retainDays, so the local directory holds a rolling window, not the whole history. A deletion that fails is logged and retried at the next rotation or start; it never stops the writer. Each stream reserves every <audit.path>.<sequence> name, plus <audit.path>.lock, <audit.path>.seq, <audit.path>.seq.tmp, and <audit.path>.torn, and a path whose file name ends in one of those suffixes is refused at startup, so no stream can be pointed at another’s file. The .seq file records the next sequence at every start and before every rotation, so a restart after retention or a shipper removed every sealed file continues the numbering instead of reusing a name. Read and ship the Evidence Gateway audit log describes the entry format and the stdout destination.

Ship Evidence Gateway audit records off host

The local audit file is not tamper-evident, and retention deletes it by age, so the append-only store you ship to is the audit record of the deployment. A shipper reads the same directory the runtime writes, so three rules keep it safe:

• Copy sealed files, and ship each one before audit.retainDays deletes it. The runtime never depends on a sealed file after sealing it, so removing one after it is archived is safe.
• Key archived copies on more than the file name, for example the host and the first entry’s time. Sequence numbers continue across restarts only while <audit.path>.seq survives, so a fresh or restored directory can seal a file under a name already shipped.
• Never point a shipper at the active file in a way that renames, truncates, or rewrites it, and never touch <audit.path>.lock or <audit.path>.seq. Ship <audit.path>.torn, when one appears, like a sealed file, then remove it. Any external replacement or modification of the active file makes readiness and later appends fail closed. Re-applying its owner-only mode, ACLs, or extended attributes, as configuration management does, is not a modification.

Archive the oldest sealed files first, for example:

cp /var/lib/registry-evidence/audit/evidence.jsonl.00000001 "$ARCHIVE/<host>-<first-entry-time>.jsonl"

Alert on the shipper, not on the local directory: its size stays bounded by retention even when a shipper has stopped, and local files that retention deletes before shipping are lost.

Render retention

Render keeps no application database, no source data, and no rendered document beyond the response it returns; a render is a pure function over a sealed template bundle and the request body. Its only durable, product-owned state is the sealed bundle on the operator host and the audit file described in Durable state and externally retained records.

The runtime recognizes a sealed file by its <audit.path>.<sequence> name and never deletes the active file. Retention deletes only sealed files whose last modification is older than audit.retainDays, so the local directory holds a rolling window, not the whole history. Ship sealed files the way Ship Evidence Gateway audit records off host describes; the mechanics are identical.

A release before 2026-09-25 kept its own keyed hash chain and exposed registry-render audit-verify to check it, and its runtime file accepted directory, integrityKeyRef, and maxSegmentBytes. The shared-writer release refuses all three as unknown fields and has no audit-verify, since the new log carries no chain to verify. Before the first start on the shared-writer release, archive the old chained log to append-only storage and point audit.path at a fresh file in a directory that holds no old segments; do not point the new runtime at the old chained file or directory.

Upgrade Casework and Scheduling audit

Before upgrading from the PostgreSQL audit-outbox releases, stop new traffic and keep the previous runtime running until its publisher has delivered every pending audit row. Casework migration 17 and Scheduling migration 8 refuse to drop an outbox that still contains unpublished rows, and caseworkctl plan and schedulingctl plan report that refusal before an apply. Check the pending count with your database tooling:

-- Casework database
SELECT count(*) FROM casework_audit_outbox WHERE published_at IS NULL;
-- Scheduling database
SELECT count(*) FROM scheduling_audit_outbox WHERE published_at IS NULL;

When the applicable count reaches zero, stop the old runtime, archive its audit file and sealed segments, and configure a fresh file path for the new runtime. Run caseworkctl apply or schedulingctl apply with the new release and the deployment’s runtime configuration. If apply reports unpublished audit records, restart the previous release to finish publishing and retry; do not delete pending rows to bypass the refusal. Preserve the old archive outside the new writer’s retention directory.

The new release writes directly to the shared per-process writer. Its audit entries are not chained. Collect the service files and operator companion files, and use append-only storage for tamper evidence. See Casework upgrade steps for the Casework procedure and command options.

Base Registry Engine retention

A Base Registry Engine registry keeps record history and the webhook outbox in PostgreSQL. Its audit stream is separate: back up the database and ship the per-process audit files, including operator companion files, through your log pipeline. A database restore does not restore or erase those files.

registry_internal.registry_revisions holds the retained revisions of a record and registry_data holds its current row. bregctl history erase is the only command that removes retained revisions. It erases them for one record up to a revision number, scrubs the affected correction context and retained outbox payloads, and replaces the affected cached idempotency responses with refusal tombstones, so a retried mutation key cannot replay an erased body. It leaves the current record, every later revision, and the audit journal in place. Snapshot references at or after the earliest erased commit become unavailable until bregctl history rebaseline writes a new baseline commit.

Change-request detail is separate from record history. bregctl request-retention erase removes the proposal and target snapshots, request revision snapshots, idempotency results, outbox payloads, and intake rows of one proposal version, and leaves the value-free lifecycle rows of the request. It applies only where the request entity declared retention.mode: operator_erase; the default, retain, keeps every proposal version and refuses operator erasure.

Audit retention uses the shared file writer described in Audit retention. History erasure leaves those files untouched. The database records committed erasure coverage in a dedicated table, so deleting an old audit segment does not affect successor-package admission.

The webhook outbox also has scheduled payload expiry. The runtime clears a retained payload on the first successful delivery, and clears an undelivered payload once its expiry has passed, which is also the point after which that delivery can no longer be replayed. The delivery rows themselves stay; only the payload goes.

An entity that declares accessLog writes one row per logged record read to registry_internal.registry_subject_access_log. Each row takes its expiry from the entity’s retentionDays (default 90) when the read happens, and the subject’s access-log route hides it once that expiry passes. A background job erases expired rows every minute in batches of 1000 until none remain, up to 100,000 rows a minute; a run that stops at that bound logs a warning with the remaining backlog. The job keeps running after a later package turns accessLog off, and a package change never extends an existing row’s expiry. Database backups keep their own copies of these rows.

Import and export checkpoints live on the operator host, not in the database. bregctl data import writes a checkpoint file beside a .state sidecar it owns, and bregctl data export writes its output file beside a checkpoint that binds the package revision, schema fingerprint, entity, profile, and operation. Both refuse a run whose files no longer match rather than resuming it, so keep each pair together and unedited or start both afresh. An export output holds record values and needs the handling any other extract gets.

Package files are operator-owned and nothing in Base Registry Engine expires them. Keep the project sources, module locks, generated files, test receipt, signature documents, and runtime file for every activation, and store secrets separately.

Relay recovery state

Relay V2 keeps no PostgreSQL-backed correctness state. It has no consultation lane, dispatch fencing, quota-bucket persistence, materialization publication history, batch-child replay, or audit-pseudonym keyring; those were Relay V1 concepts and none of them apply to Relay V2. A source is either an immutable snapshot file or a single read-only transaction against a live source, and Relay V2 keeps no separate database of its own. Its only durable, Relay-owned state is the package, the runtime file, the bound source path, and the audit sink already described in Durable state and externally retained records.

Because a package is not signed and carries no anti-rollback ratchet, Relay V2 has no rollback boundary to preserve across a recovery. Recovery is the same procedure as an ordinary deployment change: install the reviewed package, source, and runtime file at new revisioned paths and replace the complete revision. See Replace complete revisions for the exact steps.

What remains outside the product

Registry Stack does not implement a general erasure workflow for source records, audit records, or cache entries, beyond the bounded Base Registry Engine commands described in Base Registry Engine retention. It also does not prune operator-owned config files, source files, or old release artifacts after the product-level expiry checks have run. Those controls belong in the operator’s storage, backup, incident-response, and data-protection processes.

Next

• Operate Registry Relay: replace complete revisions
• Retain, erase, and audit for the Base Registry Engine erasure, rebaseline, and audit commands
• Harden a production deployment
• Known limitations and non-guarantees