Released docs. You are viewing the documentation published with v0.34.0. Development docs are available at Latest.
You operate a running Registry Casework deployment, hold its migration database credential, and
need to account for what the deployment keeps, remove the local payload copies a source owner has
authorized you to remove, or close out a source attempt whose outcome Casework could not observe.
Casework expires unified review state on its policy clocks and keeps source-backed work until you act. The
operator commands here each change one exact selection, take no copy first, and write nothing until
you repeat them with --apply.
What Casework retains
Section titled “What Casework retains”Two retention regimes run side by side. Unified reviews carry declared periods and the runtime enforces them. Source-backed work carries no clock at all: it stays until an operator erases it for one exact source request.
| State | Kept for | What removes it |
|---|---|---|
| Review request context, result and constraints, notes, drafts, tasks, and minimized history | terminalDays from terminal settlement | The runtime retention pass |
| Review clock occurrences, including a subject clock a changes-requested settlement paused | The same terminalDays from terminal settlement of the round the clock is bound to; a later round of the same review kind inside that window continues the subject clock and takes over the binding, and a round after it starts a fresh subject clock with a full deadline | The runtime retention pass |
| Producer result-feed events | The same terminalDays from terminal settlement | The runtime retention pass |
| Protected accountability record: the raw issuer-qualified deciding identity, the staff reason, and only a sha256 digest of a recorded result | accountabilityDays from the same terminal time, never shorter than terminalDays | The runtime retention pass |
| The request’s producer and initiator identities | Raw until terminalDays, then a request-bound sha256 tombstone (not keyed with a secret) until the accountability deadline, so the same producer or initiator still learns the result expired | The runtime retention pass |
| Review-create and mutation replay responses | Until request payload expiry, then a payload-free tombstone until the accountability deadline | The runtime retention pass, after which the key is free again |
| Source-backed subjects, work items, private drafts, attempts, receipts, history, and durable events | No expiry; they stay for the life of the database | caseworkctl retention erase --apply |
| The keyed audit JSONL file and its sealed segments | Your retention policy alone | No Casework command |
Each review kind declares terminalDays and accountabilityDays in the project, and the project
is refused unless terminalDays is at least 1, accountabilityDays is at least terminalDays,
and accountabilityDays is at most 3,650. Pin the two periods to the institution’s own decision
record: Casework enforces the numbers it was given and holds no view on what they must be.
The retention pass runs inside casework serve, on every thirtieth tick of the two-second
maintenance loop, so about once a minute. It works in bounded batches of at most 100 rows per
table per pass, skipping locked rows, which keeps it from competing with live traffic and means a
large backlog drains over several passes rather than in one. It deletes expired result-feed events,
request context, results, notes, drafts, tasks, history, clock occurrences, and replay
responses, then deletes the
protected accountability record and payload-free replay tombstones at their later expiry. A failed
pass logs a warning and the next tick retries.
A structured decision result is decision data, not source record content, but it follows the same
clock as the request context it was decided on: the result is deleted at terminalDays, together
with the constraints the producer supplied at create time. What outlives them is the accountability record’s sha256 digest of the result,
which proves that a specific result was decided without keeping its content until
accountabilityDays closes.
Producer clients must poll inside the terminal period, because an expired request is no longer
reachable through the request read, result, history, notes, tasks, or result feed. When a
review request payload expires, Casework erases the stored idempotency response and keeps only a
request hash and hashed binding metadata in a tombstone. An exact retry against that tombstone
returns the idempotency.expired problem with HTTP 410, so the caller reconciles the original
operation instead of creating a second request; a changed request under the same key still returns
idempotency.key-reused. Once the accountability deadline passes, the tombstone is forgotten and
the key may be reused.
Before you run these commands
Section titled “Before you run these commands”Run these commands on the operator host, where the secret root and the migration database
credential are readable. Each one loads runtime.yaml in the project directory unless you pass
--runtime-config. That file’s package.root selects the authored casework.yaml; inspect it
before applying a command. Each connects with the migration credential, not the runtime credential.
Each command previews by default and writes nothing until you repeat the exact invocation with
--apply. These examples use --format json so you can compare the exact selector, counts, and
applied state in the preview and apply reports. A successful run
exits 0. A refusal puts {"ok": false} with a diagnostic on standard output and exits 1 for an
invalid project or configuration, or 3 for an operational failure. Without --format json,
Casework prints a human-readable result and sends refusal diagnostics to standard error.
Erase source-backed payload copies
Section titled “Erase source-backed payload copies”A source owner decides what an authoritative record keeps. This command removes only the copies Casework holds locally about one exact source request, named by its source identifier, request kind, and request identifier. Preview it first:
caseworkctl --format json retention erase /etc/registry-casework/package \ --runtime-config /etc/registry-casework/runtime.yaml \ --source-id professional-register \ --request-kind "<request-kind>" \ --request-id "<request-id>"The report is counts and the selector you passed, with no work item identifiers and no erased
values: items, drafts, correctionContexts, attemptPayloads, receiptPayloads,
historyDetails, eventDetails, idempotencyResponses, auditRecords, clockOccurrences, and
clockPreviews, beside blockedLiveAttempts and applied. Read blockedLiveAttempts before
anything else: a nonzero count means a pending or uncertain source attempt is still open on this
request, and the apply run will refuse rather than erase, so that Casework never buries an action
whose outcome nobody has established. Recover or settle that attempt first. Then check the counts
against the source owner’s retention decision.
The apply run removes the local payload bytes for good and takes no copy first. Export whatever the institution’s policy keeps before you run it, and confirm the selector names the request the source owner authorized: the report shows counts, so a wrong selector looks like a plausible result rather than an error. Casework holds no way back, and a source-side copy is not one either.
caseworkctl --format json retention erase /etc/registry-casework/package \ --runtime-config /etc/registry-casework/runtime.yaml \ --source-id professional-register \ --request-kind "<request-kind>" \ --request-id "<request-id>" \ --applyOne transaction marks the local subject inactive and the items erased, deletes the private drafts
and the correction context, strips the decision reason, flagged fields, displayed binding,
recovery evidence, and receipt metadata from settled attempts, empties the detail of every history
row and durable event row, drops the stored idempotency responses, clears the retained detail,
reason, and source receipt from the audit outbox rows, and cancels the outstanding clock work for
that request. It keeps the rows themselves as bounded tombstones, so the item lifecycle stays
accountable while its values are gone, and it appends one casework.source_retention_erased audit
record carrying the selector and the same counts. The apply report repeats the preview counts with
applied set to true.
Erasure stays local to this deployment: the command reaches no Base Registry Engine (BReg) instance, sends no request to the source, and leaves the audit JSONL file as it is, so any detail already published to that file stays there until your own retention policy removes it.
Verify and recover
Section titled “Verify and recover”A completed apply run exits 0 and reports applied: true. Re-running the preview afterwards is the
check: the payload counts come back as zeros, because nothing selectable is left to erase.
A refusal names its cause without naming the request. a source attempt is still pending recovery
means an attempt on this request is pending or uncertain; recover it through the holder’s client,
or settle it, then run the erasure again. the requested resource was not found means no local
subject matches the three selector values, which is the usual shape of a typo in one of them.
the request is invalid means one selector value is empty, longer than 512 bytes, or carries a
control character. If the selected package is unexpected, inspect package.root in the selected
runtime configuration and pass the intended file.
Settle an uncertain source attempt
Section titled “Settle an uncertain source attempt”An attempt turns uncertain when Casework sent a decision to the source and could not confirm what happened to it. The holder’s client recovers such an attempt by replaying it, and periodic readback of the source repairs most of the rest. What remains is the case where neither can observe the outcome. Establish with the source owner what the source actually did, then preview the settlement:
caseworkctl --format json attempt settle /etc/registry-casework/package \ --runtime-config /etc/registry-casework/runtime.yaml \ --attempt-id "<attempt-id>" \ --outcome not-applied \ --reason "<what the source owner confirmed, and how>" \ --decided-by "<who decided>"The attempt identifier is the one Casework returned in the Registry-Casework-Attempt header
beside the work-item.recovery-pending problem the holder saw. Use --outcome applied when the
source applied the decision and --outcome not-applied when it did not. The preview takes the same
locks and runs the same checks as the settlement, then rolls back: it reports attemptId,
itemId, operation, bindingReference, your outcome, reason, and decidedBy, and the
attemptState and itemState the settlement would produce, with applied still false. The
bindingReference is a sha256: digest over the subject and the displayed binding, not a source
value.
--reason accepts at most 2,000 bytes and --decided-by at most 256 bytes; both must be non-empty
text without control characters. Both are recorded verbatim in the work item’s history, which every
Staff member and Supervisor serving that queue can read, so write them for that audience.
Applying moves the attempt out of the uncertain state permanently, and the history event stating
the outcome, the reason, and the decider stays on the work item. Settle only what the source owner
has confirmed. A wrong applied leaves the work item waiting for a source revision that never
arrives, until the next readback observes the source and returns the item to its queue, which
discards the holder’s claim.
caseworkctl --format json attempt settle /etc/registry-casework/package \ --runtime-config /etc/registry-casework/runtime.yaml \ --attempt-id "<attempt-id>" \ --outcome not-applied \ --reason "<what the source owner confirmed, and how>" \ --decided-by "<who decided>" \ --applyOne transaction changes the attempt, its work item, and the item’s history together.
not-applied refuses the attempt and returns the work item to its holder, who can act on it again.
applied completes the attempt with no source receipt and leaves the work item synchronizing, and
marks the subject for synchronization so the next source observation reconciles it. Either way the
run writes a fresh execution token, which fences any executor still holding the old one, and
appends one attempt_settled history event carrying the attempt, binding reference, operation,
outcome, reason, and decider, with no actor, because a settlement is an operator act rather than a
caseworker’s decision. This command reaches no BReg instance.
Verify and recover
Section titled “Verify and recover”A completed apply run exits 0 and reports applied: true with the states the attempt and the item
now hold. Confirm the same states through the work item’s history, where the attempt_settled
entry now sits.
Four refusals cover the cases worth planning for. the source attempt still holds a live execution lease; wait for it to expire means a client acquired the attempt within the last 330 seconds and
may still be completing it; wait, then preview again. the source attempt is completed (or
refused, or pending) means only an uncertain attempt can be settled, and a terminal attempt
already has its outcome. Recover a pending attempt first: once its execution lease expires, the
actor who started it calls POST /v1/work-items/{itemId}/attempts/{attemptId}/recover while the
source is reachable. Recovery completes or refuses the attempt from the source’s answer, and leaves
it uncertain only when the source cannot give one; settle it only then. When that actor cannot
recover it, mark it uncertain first. no source attempt has this identifier means the UUID names nothing in
this database. the work item is claimed (or another state) means the item is no longer awaiting
its source outcome, so recovery or readback has already resolved it and no settlement is needed.
Mark a stranded pending attempt uncertain
Section titled “Mark a stranded pending attempt uncertain”Only the actor who started a pending attempt can recover it: the recover route is bound to that actor’s issuer, subject, and Casework profile, and no Supervisor or Administrator can recover it in their place. When that actor cannot, because they left or their client is gone, the attempt stays pending and its work item stays synchronizing. Once the attempt’s 330-second execution lease has expired, mark it uncertain and then settle it as above:
caseworkctl --format json attempt mark-uncertain /etc/registry-casework/package \ --runtime-config /etc/registry-casework/runtime.yaml \ --attempt-id "<attempt-id>" \ --reason "<why the actor who started it cannot recover it>" \ --decided-by "<who decided>"The preview takes the same locks and runs the same checks as the marking, then rolls back. It
reports attemptId, itemId, operation, bindingReference, the originalActor issuer and
subject with their originalProfileId, your reason and decidedBy, and the attemptState and
itemState the marking would produce, with applied still false. --reason and --decided-by
carry the same limits as a settlement and land in the same work item history, so write them for
the Staff members and Supervisors serving that queue.
Repeat the run with --apply. One transaction moves the attempt to uncertain, writes a fresh
execution token that fences the original executor out, leaves the work item synchronizing, and
appends one attempt_uncertain history event with no actor. Its detail carries the attempt,
binding reference, and operation, your text as operatorReason and decidedBy, and the
originalActor and originalProfileId, so the history names both the operator’s decider and the
person whose attempt it was. The marking decides nothing about the source outcome and reaches no
BReg instance: settle the attempt once the source owner confirms what the source did.
the source attempt still holds a live execution lease means the lease has not expired yet; wait,
then preview again. the source attempt is uncertain; only a pending attempt can be marked uncertain means the marking is not needed; settle the attempt. completed or refused in its
place means the attempt already has its outcome. the work item is claimed (or another state)
means the item no longer awaits its source outcome, so nothing is stranded.
Keep the audit file
Section titled “Keep the audit file”Casework appends every audited event to a keyed hash-chained JSONL file through a single-writer
sink. Once the active file reaches 10 MiB, the sink seals it under an ascending numeric suffix
(casework.jsonl.00000001, casework.jsonl.00000002, and so on) and opens a fresh active file at
the configured path; no sealed segment is ever deleted. Every segment is yours to hold: its
lifetime, its off-host copies, and its disposal are your retention policy alone, and no Casework
command prunes, exports, or erases any of them, including the two here. Ship segments off host when
completeness matters outside one machine’s disk.
Casework 0.33.0 and earlier rotated the file instead, to .1 through .49, and deleted the
oldest. This release neither reads nor continues that layout: before its first start, move the
audit file and its numbered siblings into an archive directory, and keep that archive under the
same policy and the same audit key. Upgrade from a release that rotated the audit
file gives the steps.
The chain is keyed with the audit secret named by the runtime file’s audit.hashKeyRef, which
authenticates the retained file against a writer that cannot read the deployment secret. Provide at
least 32 bytes of NUL-free text through a secret provider, never in the runtime file:
(umask 077 && openssl rand -hex 32 > /run/secrets/registry-casework/casework-audit-key)chmod 0400 /run/secrets/registry-casework/casework-audit-keyA file secret sits immediately below the configured secret root, belongs to the runtime user, and
carries mode 0400 or 0600 with no symlink and no second hard link. Its exact bytes are the key,
including a trailing newline, so rewriting the file with a different byte breaks the chain the
retained file was written under. An empty value, a value over the provider’s size limit, or one
carrying a NUL byte is refused without echoing it. caseworkctl doctor resolves the reference as a
readiness check and reports success without printing the bytes.
Publication runs as its own loop, once a second, moving audited events from the database outbox to
the file and marking each one published. A pass that cannot read the outbox, append to the file, or
mark an event published makes GET /ready fail while the process keeps serving, and logs one
warning naming the failed stage and nothing else. A later successful pass restores readiness and
logs the recovery. Treat a /ready failure with that warning as a disk, permission, or lock
problem at the audit path rather than a database outage, and check that no second writer holds the
sentinel lock beside the file.
Troubleshooting
Section titled “Troubleshooting”| Symptom | Next move |
|---|---|
The erasure preview reports a nonzero blockedLiveAttempts | A pending or uncertain attempt is open on that request. Recover it through the holder’s client, or settle it, then preview again. |
retention erase --apply refuses with a source attempt is still pending recovery | An attempt reached one of those states between your preview and your apply run. Resolve it and repeat both runs. |
retention erase refuses with the requested resource was not found | No local subject matches all three selector values. Confirm the source identifier, request kind, and request identifier with the source owner. |
| A command inspects an unexpected Casework package | Check package.root in the selected runtime configuration and pass the intended --runtime-config file. |
attempt settle refuses with the source attempt still holds a live execution lease | A client acquired the attempt within the last 330 seconds. Wait for the lease to expire, then preview again. |
attempt settle refuses with only an uncertain attempt can be settled | The attempt is pending, completed, or refused. A terminal attempt already carries its outcome. Recover a pending one first: once its 330-second execution lease expires, the actor who started it calls POST /v1/work-items/{itemId}/attempts/{attemptId}/recover while the source is reachable. Settle only if recovery leaves it uncertain. If that actor cannot recover it, mark it uncertain with attempt mark-uncertain, then settle it. |
attempt mark-uncertain refuses with only a pending attempt can be marked uncertain | An uncertain attempt needs no marking; settle it. A completed or refused attempt already carries its outcome. |
attempt settle refuses with only a work item awaiting its source outcome can be settled | Recovery or source readback already moved the item out of synchronizing. No settlement is needed. |
| A review request disappeared from producer polling and the result feed | Its terminalDays period closed. The accountability record may still exist; an authorized Supervisor can resolve one event until accountabilityDays closes too. |
An exact retry returns idempotency.expired with HTTP 410 | The item payload expired and only a payload-free tombstone remains. Reconcile the original operation before choosing a new key. |
GET /ready fails with an audit publication warning in the log | The audit path, its permissions, or its single-writer lock is the likely cause. Fix the path and let the next pass restore readiness. |
- Run Registry Casework for the deployment this page maintains.
- Author a Casework policy for the review kinds that declare
terminalDaysandaccountabilityDays. - Retention and persistent state for what the other Registry Stack runtimes retain.
- Harden a production deployment for the controls around the migration credential and the secret root these commands rely on.
- Read the Casework API for the problems and headers named here.