Skip to content
Registry StackDocsv0.46.0

Operate the Registry Evidence audit log

For the operator

View as Markdown

You run a deployed Registry Evidence service and answer for what it disclosed. For every admitted operation, Evidence writes one entry before any source read and one with its outcome, and refuses the request when an entry cannot be written. Requests refused before admission write nothing: authentication failures, malformed requests, invalid selectors, and rate-limited requests. The log is tamper-evident only once it reaches the append-only storage you ship it to.

You author the audit block in the deployment target, not in the runtime file. The production example target declares:

audit:
hashKeyRef: secret:file/evidence-audit-hmac
hashKeyVersion: 1
path: /var/lib/registry-evidence/production/evidence.jsonl

evidencectl package splits the block. audit.hashKeyRef and audit.hashKeyVersion name the audit master and its version, and go into the governed package. audit.destination, audit.path, audit.rotateBytes, and audit.retentionDays go into the candidate’s runtime.yaml:

  • destination: file, the default, appends to path and acknowledges an entry only after it is synced to disk. It rotates the active file at rotateBytes (100 MiB by default) and deletes sealed files older than retentionDays (90 by default).
  • destination: stdout writes one flushed line per entry on standard output, and refuses path, rotateBytes, and retentionDays: durability, rotation, and retention belong to your collector. Operational records go to standard error.

One process writes one file. The writer holds <path>.lock for its whole life, so give each replica its own path or use stdout. The directory must belong to the service user and be unwritable by group or others.

Before you start a candidate, prove its audit file sits on your persistent storage:

Terminal window
evidencectl doctor \
--runtime-config /srv/registry-evidence/candidate/runtime.yaml \
--require-audit-under /var/lib/registry-evidence

When every dependency is reachable, doctor prints Dependency preflight passed for the runtime file and a Proof: line. When the root does not exist on the host, it refuses with evidence.deployment.audit-outside-root and exits 3. A stdout destination is refused here, because it has no local file.

A candidate staged beside the instance it replaces meets the running writer’s lock:

Evidence runtime dependency preflight refused <runtime-file>
error[evidence.deployment.audit-unavailable] <runtime-file>
runtime audit initialization failed: another process holds the single-writer lock beside the audit file; stop it before starting this one
next: Make the audit destination writable by this user and free of another writer, as the message says.
1 error, 0 warnings in <count> files
Next: Another Evidence instance holds this audit destination's writer lock. To check a candidate staged beside it, rerun doctor with --without-audit-lock; otherwise stop the other writer first.

Rerun with --without-audit-lock. Doctor still checks the destination’s ownership, modes, write access, and final entry, and its Proof: line says a second writer is not detected. See evidencectl doctor for every option.

Tamper evidence comes from where entries end up, not from the file the service writes. Once the service runs:

  1. Pick a store the Evidence host cannot rewrite: object storage with an object lock, a write-once archive, or a log pipeline whose retention the Evidence operator account cannot change.
  2. With destination: file, ship every sealed file. Rotation renames the active file to <path>.<sequence>, an ascending eight-digit number, and the next number is kept in <path>.seq. Copying a sealed file is safe at any time. Ship each one before audit.retentionDays deletes it, so set that key in the target’s audit block longer than your shipper can fall behind. Key archived files by host or replica as well as by name: two replicas use the same names, and numbering restarts when <path>.seq is lost.
  3. If a crash left <path>.torn, the bytes of an unfinished entry that belongs to no acknowledged entry, inspect it, archive it with the sealed files, and then remove it. The writer refuses a later torn line while the side file is still there.
  4. With destination: stdout, point your collector at the store; each line on standard output is one entry.
  5. Verify, as part of your backup, restore, and incident procedures, that entries reach the store: pick an operation and find its entries there by their shared correlation.

Restore audit history into that store, never into the live audit directory. The operator contract is the authority for these rules.

Never rename, edit, or truncate the active file while the service runs: the writer treats that as tampering, and every later audited request fails closed until a restart. A stdout destination stalls when its collector stops reading. After the writer has held one entry for 5 seconds, audited requests are refused and readiness fails; standard error carries audit stream has held one entry past the write deadline, and later audit stream took the entry it was holding once the collector reads again. Restore the collector rather than restarting, which can lose queued entries.

Each entry is one JSON line. New authorized operations use registry.evidence.audit/v3 or registry.evidence.audit.request-batch/v3; authorization refusals keep registry.evidence.audit.authorization-refusal/v2. Retained authorized /v2 entries remain readable history. Entries are not chained. The audit-event.schema.yaml and request-batch-audit-event.schema.yaml contracts define the members and each phase’s closed decision values. The record’s phase says what the entry establishes:

Record phaseWhat it establishesWhat it does not establish
access-attemptAuthentication and the complete authority path were accepted before source credentials or data were touched. In a current /v3 record, executionBounds captures the positive operation, deadline, and response-byte limits that govern the acquisition.That the source returned a record or that anything was released. The bounds carry no endpoint, credential, secret, or source value.
disclosure-releaseThe named concepts were serialized and durably recorded before the response bytes left the service.That the source fact was true or that the caller used it correctly.
denialThe operation was refused, either as an authenticated authorization refusal or as an evaluation that decided no answer.The rejected requirement, selector, or source value behind it.
transient-failureAn admitted operation failed in one closed, value-free class a retry may clear.The selector, source value, or credential that triggered it.

Requester and subject handles are keyed pseudonyms prefixed hmac-sha256:v<version>:, derived from the audit master: protected correlators, not anonymous values. A record never carries a raw selector, a released or source value, or an access token. No supported tool turns a person’s identifier into their access history, so do not expose the audit master or raw records to build one. On a local development session, evidencectl audit show reads the last operation, as Stop and read the audit shows.

The closed executionBounds object appears only on current /v3 access attempts. Its members are maximumDataOperations, acquisitionDeadlineMilliseconds, and maximumResponseBytes. Evidence captures the effective values from the immutable runtime configuration before credential resolution or source effects. When sourceAcquisition is omitted, the values are 5, 30000, and 1048576. Terminal /v3 records and retained /v2 records omit the object. The deadline gates admission to the terminal release append, the disclosure-begin boundary. If that append is admitted before expiry, it completes without cancellation and the exact audited bytes may return after expiry; this is not a hard wall-clock response cutoff. A signed source’s verified validUntil remains an independent data-validity bound.

The unauthenticated public check route, POST /v1/public-check, writes its own entries. Each one is a platform audit entry with the schema registry.evidence.audit.public-check/v1, and the public-check-audit-event.schema.yaml contract defines it. The route has no requester, actor, client, or grant, so no record names one, and none carries a response protection, an Evidence identifier, or a signing key, because the public answer is never signed. A check refused before admission (a malformed body, a document number that fails the selector profile or the Source inputs schema, or a rejected request limit) writes nothing. The entry phase is request for the access attempt and response for every other phase, and the entry correlation is the record’s operation.

Record phaseDecisionWhat it establishes
access-attemptauthorizedThe subject resolved and the record was durably accepted before the Source was read. executionBounds carries the effective acquisition budget.
disclosure-releasereleasedThe exact answer bytes were serialized and durably recorded before they were returned. disclosedConcepts names exactly the status and fields concepts.
denialno-match, ambiguous, unresolved, fact-missing, or code-mismatchThe check ended in a miss, with a safeErrorCategory.
transient-failuredependency-failure or evaluation-failureAn admitted check failed in a value-free class a retry may clear, with a safeErrorCategory.

All five miss decisions are answered to the caller with one identical 404 resource.not-found problem. The log is where you tell them apart: code-mismatch also covers a stored check code that is absent or malformed, and it is visible to you only. A document number that passes the selector profile and the inputs schema but that the Source binds to a path segment it cannot occupy is admitted, so it leaves an access-attempt record and a transient-failure record with dependency-failure, and the caller gets 503 source.unavailable; the Source is never contacted.

Every record carries assuranceProfile, eventId, occurredAt, operation, phase, requirement, bundleRevision, authorityRevision, effectiveGrantDigest, purpose, subjects, sourceId, adapterId, decision, reason, and durationMilliseconds. reason is always public-check.activated. subjects holds the one presented selector bundle as a role, a selector profile, and a keyed selectorBundlePseudonym. Repeated checks of one document number share that pseudonym, so you can see one number tried repeatedly without the log holding the number.

A record never carries the presented or stored check code, the document number or any selector value, request headers, a client address, an inbound trace context, a source value or fact, or a credential. An audit append failure answers 503 service.unavailable and releases nothing.

Every pseudonym carries audit.hashKeyVersion in its prefix, so entries under different masters stay distinguishable in one log and rotation needs no fresh audit path.

  1. Generate a fresh, independent master into a new secret file (written 0600):

    Terminal window
    evidencectl keygen secret --output /run/registry-evidence/secrets/evidence-audit-hmac-2
  2. In the deployment target, set audit.hashKeyRef to secret:file/evidence-audit-hmac-2 and raise audit.hashKeyVersion by one. Record both versions in your change record.

  3. Repackage the deployment target into a new candidate and install it with the target-host steps in Build and deploy a Registry Evidence project. Route traffic only after readiness succeeds.

  4. Keep the previous master under its secret controls for as long as pseudonyms from its version must stay recomputable for an investigation.

The audit master must differ from the subject-binding secret and the signing key.