Refresh and recover a materialization

View as Markdown

Use this procedure to refresh, investigate, retain, or recover a Registry Relay materialization without changing the source contract or enabling request-time source fallback.

Prerequisites

Identify the logical entity, physical environment binding, current generation, refresh mode, and freshness contract.
Know the protected Relay admin listener and the least-privilege reload and posture credentials.
Preserve the source file, Relay cache, Relay consultation database when enabled, configuration, secrets, trust anchor, and anti-rollback state under the deployment retention policy.
Block dependent traffic before a recovery that can change the published generation.

Ownership and trust boundary

The entity owns the logical schema, primary key, maximum records and bytes, refresh policy, and retained-generation intent. The environment owns the private provider path, physical columns, source revision, and generation. Registry Relay owns ingest, immutable publication, readiness, and last-good serving.

An exact snapshot consultation reads the published handle and does not open a live source connection during the request. If the required materialization is absent, stale, or inconsistent, the dependent profile becomes unready. Registry Relay does not fall back to live source access.

Inspect before refreshing

Read liveness, readiness, and redacted posture separately. Use restricted posture only on the protected admin network when the operator needs per-resource refresh observations.

The restricted relay.refresh_health observations report the last successful load, consecutive refresh failures, and whether Relay serves last-good data. They report acquisition health, not the source record’s business freshness.

Review the Relay operations runbook for the current admin listener and scope requirements.

Refresh the materialization

Registry Relay supports mtime, interval, and manual refresh modes. For a manual table reload, call the protected table endpoint:

curl -X POST \
  -H "Authorization: Bearer <relay-admin-token>" \
  http://127.0.0.1:8081/admin/v1/datasets/<dataset-id>/tables/<table-id>/reload

The table-specific endpoint runs the bounded acquisition and durable materialization-publication path for that one resource. On failure, ordinary reads retain their previous ready table when one exists. If durable SnapshotExact publication already advanced before local registration failed, Relay removes that publication slot from consultation use. Global /ready can remain 200, but every dependent SnapshotExact consultation fails closed until exact durable reconciliation succeeds.

Do not use /admin/v1/reload for a deployment that contains any audited SnapshotExact plan. Current Relay rejects the complete reload-all request with ingest.materialization_failed, marks every resource failed in that report, and refreshes no resource. There is no atomic multi-materialization admin refresh in current source. When several materializations must move together, block dependent traffic, refresh each exact table through its table-specific endpoint, verify every intended publication, then readmit traffic. If the deployment requires atomic publication across several materializations, keep traffic blocked and escalate because current Relay cannot establish that property.

After a successful reload, require the dependent readiness and posture observations to refer to the intended generation before admitting traffic. Run an authorized exact-lookup canary with synthetic identifiers when the deployment permits that probe.

Handle stale or failed refreshes

A refresh failure after a successful ingest leaves the last-good table active and can leave /ready at 200. The last successful load timestamp does not advance, and consecutive failure observations increase. A resource with no successful generation remains not ready.

Treat these states differently:

ready with serving_last_good: true means the process can serve the retained generation. It does not mean that the source owner published current business data.
An ordinary last-good table can keep global /ready at 200 after refresh acquisition fails. This is an availability state, not SnapshotExact freshness evidence.
pdp.evidence_stale means the policy rejected evidence age. Do not weaken the policy or rewrite the timestamp to clear the denial.
schema.resource_unavailable means the configured resource cannot serve. Inspect ingest health and the stable error reference before changing traffic.
Execution-time SnapshotExact freshness is checked during each request. A stale publication returns consultation.unavailable for that execution while global /ready can remain 200. A digest-inconsistent or incomplete publication slot also stays unavailable. Do not enable a live connector as an automatic fallback.

Use the error and status code reference and operator diagnostic reference for stable classifications.

Retain materialization state

Relay’s ingest cache can contain complete normalized source rows. Protect the cache with the same data classification as the source. For audited SnapshotExact, the entity’s authored retain_generations value keeps from 1 through 16 completed cache generations, including the active generation, after successful publication. Ordinary resources keep the built-in current and immediately previous snapshot. Relay removes older cache files on a best-effort basis. There is no time-to-live for the ingest cache. The bounded retained set supports recovery and readers. It is not an admin-selectable list of arbitrary rollback targets.

Consultation materialization publication history is separate durable PostgreSQL correctness state. Registry Relay does not provide general time-based pruning for that history. Back up immutable source inputs, the Relay ingest cache, and the complete Relay consultation database at one coordinated quiesced recovery point. Preserve role bindings, audit-pseudonym key material, and the audit watermark. The recovery evidence must bind the source archive, cache archive, database active pointer and history, and exact active generation and restricted content digest.

The retention and persistent-state reference defines the complete store inventory.

Expected evidence

Retain:

The previous and candidate generation identities without source rows.
The reload result and per-resource status.
The last successful load time and consecutive failure count.
Health, readiness, audit, and redacted posture after publication.
The exact-lookup canary outcome and the traffic-admission decision.
One attested recovery record binding source inputs, cache, Relay database active pointer and history, exact generation, restricted content digest, and recovery-point metadata.

What this proves

A successful table-specific reload proves that Relay read, validated, and published that configured source resource under the active runtime contract. Readiness proves that the required runtime dependencies accept traffic at the time of the probe. An exact synthetic canary proves the selected published path.

These checks do not prove source-domain freshness, completeness, legal suitability, country acceptance, off-host audit receipt, or every record. Readiness cannot detect writes missing from a stale database restore.

Roll back or recover

A failed table-specific reload preserves that resource’s last-good generation when one exists. It does not provide an atomic rollback boundary across several materializations. Current docs do not define an admin operation that selects an arbitrary older cache generation. Do not edit cache files or publication rows to simulate rollback.

For recovery, keep traffic blocked and restore the complete matching recovery set, or republish a reviewed source export with a new environment generation and repeat the offline, reload, posture, and canary gates. When consultation state may be stale, recover forward to the last acknowledged write and external audit watermark before resuming the same workload.

Escalate

Escalate to the source owner when business freshness or completeness cannot be established. Escalate to Relay operations when reload-all cannot retain the previous coherent generation, cache integrity is uncertain, publication history and cache disagree, or the recovery point may omit an acknowledged consultation. Escalate to the security owner before moving or exposing a cache that contains source rows.