Skip to content
Registry StackDocsv0.34.0

Retain, erase, and audit

For the operator

View as Markdown

You operate an active registry under an institutional retention policy, hold the migration credential, and need to erase retained history or change-request detail, restore snapshot coverage after an erasure, or prove, copy, and prune the audit journal. At the end of this page each of those maintenance actions has run once, under the interlock the migration authority uses, with a chained audit record and a report that carries counts rather than values. The erasure, attachment cleanup, and pruning commands remove bytes that nothing restores. The next section says exactly what each one destroys.

CommandRemoves for goodLeaves in place
history eraseThe retained revisions of one record up to a revision number, the correction context and retained outbox payloads that referenced them, and the cached idempotency responses that could replay them.The current record, every later revision, and the audit journal.
request-retention eraseThe payload detail of one change-request proposal version: proposal and target snapshots, idempotency results, request revision snapshots, outbox payloads, intake rows, and attachment references/content.The value-free lifecycle rows of that request, and record history.
request-retention cleanup-attachmentsUnreferenced external attachment bytes awaiting deletion, including abandoned uploads and objects recreated after an earlier confirmed deletion.Live attachment references, request detail, and durable deletion tombstones.
evidence-retention erase-expiredThe signed Evidence responses and verification context a governed action retained, once their 24-hour window has closed.Every retained assertion still inside its window, and the action receipts that used them.
audit pruneThe longest run of audit records at the start of the chain whose creation time precedes the boundary you give.Every later record, and one retention record stating what left the chain.

history rebaseline, request-retention list, request-retention dry-run, audit verify, and audit export remove nothing: the rebaseline writes one baseline commit, the export creates one file, and the others only read. A saved export or a snapshot reference is not a way back either: an erased revision is gone from the registry, and a snapshot reference that named it can no longer be served.

Every command on this page reads the runtime file and connects to PostgreSQL directly with the migration credential, and the erasures, the rebaseline, and the prune take the exclusive registry lock, so run them from the operator host while no activation is in progress. When the database uses a private certificate authority, set SSL_CERT_FILE to its PEM bundle for each command. Paths given to --runtime-config, --request-file, and --output must be absolute.

The migration authority can erase the retained revisions of one record up to a revision number. This is irreversible maintenance, not an API permission, and it does not remove the current record. Apply the institution’s live-data removal policy separately.

Write an owner-only JSON request file (mode 0600) so identifiers and reasons never appear in process arguments:

{
"entityId": "membership-record",
"recordId": "00000000-0000-4000-8000-000000000001",
"eraseThroughRevision": 2,
"operatorReference": "approved-maintenance-001",
"reason": "approved-retention-request"
}

history erase refuses to run without --acknowledge-irreversible, because erasure cannot be undone and no command restores the erased revisions.

Terminal window
bregctl history erase \
--runtime-config /etc/breg/runtime.yaml \
--request-file /srv/registry/private/erasure.json \
--acknowledge-irreversible

One transaction erases at most 10,000 revisions, scrubs affected correction context and retained outbox payloads, and replaces affected cached idempotency responses with refusal tombstones, so a retried mutation key cannot replay an erased body. The result and the audit record carry counts, not values. A refusal names its cause without naming the record, and a request file that is not owner-only is refused before any connection is opened. Snapshot references at or after the earliest erased commit become unavailable, including every later snapshot; erasing baseline data makes all snapshot coverage unavailable. Nothing re-baselines the registry on its own: run history rebaseline when snapshot reads of current state must work again. Live reads and writes continue.

An erasure of revisions committed after the coverage baseline does not hold back the next package: bregctl apply accepts a successor when the only gap in coverage is one the audit journal records as that erasure. Erasing baseline data, or any gap the journal does not record as a committed erasure, still refuses every successor package until history rebaseline restores coverage. That includes an erasure whose audit record audit prune has since removed. The refusal is apply.history.coverage_incomplete, and it leaves maintenance state unchanged, so apply the same package again once the rebaseline completes.

Restore snapshot coverage after an erasure

Section titled “Restore snapshot coverage after an erasure”

After an erasure, a snapshot reference for current state is refused until the migration authority re-establishes a covered position. history rebaseline is that command, and it runs under the same interlock as the erasure: the configured migration role, bounded timeouts, the exclusive registry lock, a ready registry, and one chained audit record.

Write an owner-only JSON request file (mode 0600) carrying the operator reference alone:

{
"operatorReference": "approved-maintenance-002"
}
Terminal window
bregctl history rebaseline \
--runtime-config /etc/breg/runtime.yaml \
--request-file /srv/registry/private/rebaseline.json

One transaction proves the retained journal head of every live row still reproduces that row, installs one baseline commit at the head, and moves the coverage baseline to it. It restores nothing that was erased: references taken before the new baseline stay unavailable, because the bytes they named are gone. Take a fresh reference afterwards. The command refuses when coverage is already complete, when the registry is not ready, when a retained journal head is not indexed by a commit, and when a live row has no retained journal head that reproduces it. The result and the audit record carry counts and positions, not values.

That one transaction verifies at most 1,000 live rows, and a registry holding more is refused with history.rebaseline.live_rows.budget_exceeded. The limit is fixed and the refusal is not retryable: a registry over that limit cannot restore snapshot coverage with this command, so treat the code as a report that the erasure left coverage unrecoverable rather than as a transient failure.

history.rebaseline.live_rows.unverified reports one live row that its journal head does not reproduce, and names neither the entity nor the record, because operator refusals here carry no values. It stops at the first entity whose live rows and journal heads do not line up, and inside an entity at the first record identifier whose values differ, so a run that gets further has already proved every earlier entity. To find the row, read the revisions of the records you suspect with GET /v1/records/{route}/{recordId}/revisions, or compare live rows with their journal heads directly under the migration role you already hold. The usual cause is a live row that changed outside the server’s write path, so the journal never recorded the change.

Change-request proposals and application receipts keep their own retention. A request entity whose changeRequest.retention.mode is operator_erase allows the operator to erase the payload detail of one proposal version while keeping its value-free lifecycle rows; the default retain mode refuses erasure. List the retained rows, then count what one erasure would remove:

Terminal window
bregctl --format json request-retention list \
--runtime-config /etc/breg/runtime.yaml \
--request-entity placement-correction-request --limit 50
bregctl --format json request-retention dry-run \
--runtime-config /etc/breg/runtime.yaml \
--request-entity placement-correction-request \
--request-id "<requestId>" --proposal-version 3

Each list row reports the request state, whether the version is current or pinned, its retention mode, eligibleForErasure, and whether detail is already erased; a listing longer than --limit continues from the cursor it returns, passed as --after-cursor. The dry run reports the counts of proposal snapshots, target snapshots, idempotency results, request revision snapshots, outbox payloads, intake rows, and attachmentReferences that the erasure would remove, and changes nothing. Canceled drafts are eligible under the same policy even when no proposal was submitted.

Terminal window
bregctl --format json request-retention erase \
--runtime-config /etc/breg/runtime.yaml \
--request-entity placement-correction-request \
--request-id "<requestId>" --proposal-version 3

The erasure reports the same counts as the dry run. A refusal, request_retention.operation.refused, names the cause without values: the entity’s retention mode is retain, or the version is pinned. Erasing record history does not erase these rows, and erasing these rows does not touch record history.

Attachment bytes are removed after their final retained reference is erased. Other requests or proposal versions that legitimately retain identical content keep their bytes and access. Erased slot metadata keeps filled state, size, and hash while dropping content type, verification status, and upload actor/time. S3 deletion failures remain in a durable retry queue; pendingExternalDeletions reports the registry-wide outstanding count. Use attachment cleanup to retry these deletions independently of request erasure. A nonzero count means physical deletion is still pending, even though scoped download access has been revoked. externalDeletionTombstones counts confirmed S3 deletions retained for future absence checks. Continue periodic cleanup runs to recheck these keys and remove any orphan bytes from a delayed remote write. Confirmed tombstones also remain subject to the first-upload storage and verification binding. Configure backup and replication expiry separately, and keep the bucket versioning rules in Attachment storage. An external verifier may retain its own copies or logs; registry erasure does not remove them. Set that service’s retention policy as part of External attachment verification.

Use request-retention cleanup-attachments when S3 deletions remain pending or when an interrupted upload left unreferenced content. Cleanup works even when all requests use retain mode or the registry has only active drafts. It uses the migration credential and verified runtime configuration, including the pinned storage and verification bindings.

Terminal window
bregctl --format json request-retention cleanup-attachments \
--runtime-config /etc/breg/runtime.yaml

The report contains the registry-wide pendingExternalDeletions and externalDeletionTombstones counts. Repeat cleanup while deletions remain pending, and continue periodic runs to recheck confirmed tombstones for delayed remote writes. A backend failure leaves work durable for a later run. A configuration or identity refusal requires restoring the matching runtime binding before retrying. This command does not make an active draft or a retained request eligible for detail erasure.

A governed action that calls a verified Evidence capability retains what it accepted. Under handler ABI registry.action-handler/v2, each accepted assertion is stored beside the applied action: the signed response bytes, the verification context, the capability, the contract fingerprint, the trust binding, the assertion identifier, its issue, observation, and validity times, the moment the registry accepted it, and the declared maximum observation age. Subject and selector values are never retained. The runtime role holds an insert-only grant on that state, so no request path reads or deletes what it wrote.

This is a separate retention scope with its own clock. Each retained assertion expires 24 hours after the registry accepted it, erasing record history does not cover it, and nothing removes the expired rows on its own. Schedule the erasure on the operator host, under the migration authority and the exclusive registry lock the rest of this page uses.

Terminal window
bregctl evidence-retention erase-expired \
--runtime-config /etc/breg/runtime.yaml \
--before 2026-01-01T00:00:00Z

The command removes every row whose 24-hour window closed at or before the cutoff, and no row still inside its window: a cutoff later than the database transaction time reaches only expired material, and a cutoff in the future is refused. Both paths must be absolute. The report carries a count and no values. The run verifies the expected registry identity, catalog, and readiness while it holds the registry transaction lock through the deletion, and the configured runtime credential cannot perform the deletion at all. Unlike request-retention erase, which erases one exact proposal version, this erasure works from the time boundary alone.

Every write path appends one hash-chained audit record, and the journal keeps its own head. The audit commands prove that chain, take a portable copy of it, and retire its oldest part under a retention boundary. They read envelope bytes, hashes, envelope identifiers, and timestamps only, so no record value reaches the report or a refusal.

Terminal window
bregctl --format json audit verify \
--runtime-config /etc/breg/runtime.yaml
bregctl --format json audit export \
--runtime-config /etc/breg/runtime.yaml \
--output /srv/registry/private/audit-journal.jsonl

verify walks the chain from the recorded head backwards through the recorded links, never by timestamp, so a clock skew or an out-of-order insert cannot change what the journal claims. It runs in one repeatable-read read-only transaction that also compares the head and the total row count, and reports the record count with the first previous hash, the last hash, and the head hash. A chain that no longer links reports audit.chain.broken; a head row that does not match the chain reports audit.head.mismatch. export verifies the same way while it writes, and emits one envelope per line in chain order. The destination must be an absolute path that holds no file; the command creates it with owner-only permissions and removes it again when the export does not finish, so a partial copy is never left behind. Verification proves the retained journal links and matches the head; it cannot prove the journal is complete, because a pruned prefix is gone and the earliest retained record’s previous hash is the only trace of it.

prune removes the longest run of records at the start of the chain whose creation time precedes one RFC 3339 boundary. It stops at the first retained record, so a gap can never open in the middle, and it refuses a boundary later than the database transaction time. Take an export first, then count what the boundary would remove:

Terminal window
bregctl --format json audit prune \
--runtime-config /etc/breg/runtime.yaml \
--before 2026-01-01T00:00:00Z --dry-run
Terminal window
bregctl --format json audit prune \
--runtime-config /etc/breg/runtime.yaml \
--before 2026-01-01T00:00:00Z

The report names whether the run was a dry run, how many records it removed and retained, the hash the retained chain now starts from, and the first retained envelope. A prune runs on the migration connection under the exclusive registry lock with the head row locked, so no writer can append and no second operator can prune while the prefix is being decided; the runtime role keeps its insert-only grant and cannot delete a record. Every prune that removes anything appends one retention record to the journal, so the chain states what left it.

SymptomNext move
A saved snapshot returns 503 source.unavailableRetained history no longer covers it, after an erasure or an incompatible successor. Take a fresh reference.
A fresh snapshot reference is refused after an erasureCoverage ends before the erased commit. Run history rebaseline to cover the current state again; earlier references stay unavailable. It verifies at most 1,000 live rows, and refuses a larger registry outright.
history rebaseline reports history.rebaseline.live_rows.unverifiedA live row and its journal head disagree, and the refusal names no record. Read the revisions of the records you suspect, or compare live rows with their journal heads under the migration role.
history erase reports history.erase.acknowledgement.requiredThe command was run without --acknowledge-irreversible. Confirm the record and the revision boundary, then run it again with the flag. Nothing was read or erased.
history.erase.request_file.refused or history.rebaseline.request_file.refusedThe request file must be an absolute path to an owner-only regular file, not a symbolic link, holding the documented fields and nothing else.
request-retention erase is refusedThe request entity’s retention mode is retain, or the version is pinned. Run the dry run for the exact version first.
evidence-retention erase-expired removes nothingNo retained assertion has passed its 24-hour window yet, or the cutoff precedes every expiry. The count is the whole report.
audit verify reports audit.chain.broken or audit.head.mismatchThe journal no longer links to its head. Export what is still reachable, keep the database as it is, and treat it as an integrity incident.
audit verify, audit export, or audit prune reports audit.envelope.invalidA reachable journal record holds an envelope no reader accepts. Nothing is pruned. Keep the database as it is and treat it as an integrity incident, like a broken chain.
audit verify, audit export, or audit prune reports audit.records.unreachableRecords exist in the journal that the chain from the head does not reach. Nothing is pruned. Keep the database as it is and treat it as an integrity incident, like a broken chain.
audit prune reports audit.boundary.future--before is later than the database transaction time. Choose a boundary in the past.
audit export is refused with audit.operation.refusedThe runtime configuration path or --output is relative, or the database is unreachable.
audit export is refused with audit.output.exists--output already names an existing file. Choose a destination that does not exist.
audit export is refused with audit.output.not_durableThe export reached --output and the directory holding it could not be made durable, so a crash may lose it. Verify the file, or remove it and export again.
audit export is refused with audit.output.invalid--output’s parent is not an available directory, or a component of it is a symbolic link. Name the real directory that holds the export.
An operator command cannot reach PostgreSQLCheck the migration URL secret, the role names, and SSL_CERT_FILE for a private authority.