Skip to content
Registry stack docs v0 · draft
Search

Publishing pipeline

A registry’s published surface lives in two places: a static bundle of standards-facing artifacts that any downstream system can fetch from a file host, and a runtime metadata endpoint that knows who is asking and responds accordingly. Registry Manifest owns the first path. Registry Relay owns the second. The two paths share the same manifest input, but they serve different deployment needs.

Publishing pipeline. One declarative input (metadata.yaml: catalog, dataset, entity, and policy blocks; no source paths, no credentials). Two publication paths, both using the same renderers. (1) Static path via registry-manifest-cli publish emits DCAT, BRegDCAT-AP, SHACL, JSON Schema, CPSV-AP, and ODRL files with no service and no auth. (2) Runtime path via GET /metadata/* is served by a running Relay using the same renderers, filtered by the caller's scope, with per-request audit.

The two-layer split: portable vs runtime

Section titled “The two-layer split: portable vs runtime”

Portable metadata (Registry Manifest only) travels wherever files can travel. A civil registration authority can validate and publish a DCAT catalog, a BRegDCAT-AP profile, SHACL shapes, JSON Schemas, and an ODRL policy bundle from a single metadata.yaml with no running service, no credentials, and no network. The output is a directory of static files that can be served from an object store, a CDN, or a government open-data portal.

Runtime metadata (Registry Manifest renderers inside Registry Relay) is served at /metadata/* by a running Relay instance. The same renderers produce the same artifact formats, but Relay filters the view by the authenticated caller’s metadata scope, so a caller with social_registry:metadata sees only the datasets and entities their key permits. That filtering is the runtime-only capability; it cannot be replicated statically.

Both paths share the same manifest format and the same renderer code from registry-manifest-core. The static path invokes those renderers through the CLI. The runtime path invokes them at request time, inside Relay.

What Manifest emits from one YAML

Section titled “What Manifest emits from one YAML”

You write one metadata.yaml and run registry-manifest-cli publish. The CLI calls every renderer in the table on the Registry Manifest overview page and writes each output to a named file in the target directory, plus an index.json discovery entry point (schema version registry-manifest-index/v1).

A working example lives at registry-manifest/profiles/example-civil-registration/fixtures/metadata.yaml. The file’s catalog section looks like this:

schema_version: registry-manifest/v1
catalog:
  id: example-civil-registration
  base_url: https://civil-registration.example.gov
  title:
    en: Civil Registration Metadata
  publisher:
    name: Civil Registration Authority
    iri: https://civil-registration.example.gov/authority
    authority_type: eli:PublicAuthority
  conforms_to:
    - https://semiceu.github.io/BRegDCAT-AP/releases/3.0.0/
  standards:
    dcat: "3.0"
    shacl: "1.1"
    json_schema: "2020-12"

From that catalog block plus the dataset and entity blocks below it, Registry Manifest derives a DCAT catalog document, a BRegDCAT-AP profile, a SHACL shapes document, a JSON Schema per entity, CPSV-AP service catalogue entries for declared public services, and ODRL policy documents per dataset. CCCEV evidence rendering is owned by Registry Notary, not Registry Manifest. No renderer makes network calls or reads additional files.

What Relay adds at runtime

Section titled “What Relay adds at runtime”

When Registry Relay starts, it compiles metadata.yaml and validates it; it then loads the runtime config.yaml, which binds the manifest’s logical datasets and entities to actual source files or database tables, and cross-checks that every binding resolves to a declared dataset and entity. Manifest problems surface as metadata.manifest.validation_failed; binding mismatches surface as runtime.binding.dataset_missing (and related runtime.binding.* codes). Startup exits non-zero on either, so misconfiguration is visible immediately.

At request time, a caller authenticated with a <dataset_id>:metadata scope can fetch:

GET /metadata/dcat/bregdcat-ap
Authorization: Bearer <token>

Relay calls the BRegDCAT-AP renderer on the validated manifest and returns the result, which includes the CPSV public service nodes the manifest declares. If the caller’s scope covers only one dataset and the manifest has three, the response contains only the datasets visible under that scope. For the full route inventory (DCAT, SHACL, ODRL, per-dataset, per-entity, OGC Records, evidence offerings), see the Relay reference.

Beyond scoped filtering, Relay adds:

  • Per-caller views under /metadata/*. Every metadata format from the renderer table is available as an authenticated route. The same caller that can read rows with a rows scope can separately hold a metadata scope to fetch catalog artifacts.
  • Audit records for every metadata request. The same JSONL audit trail that covers row reads covers metadata reads. The caller’s principal_id, the scopes_used, and the request_id all appear in the record.
  • Scope-aware filtering. A key or token that carries social_registry:metadata but not vital-events:metadata receives a catalog that lists only the social registry dataset, even if the manifest declares both.

When to use Manifest alone vs Manifest with Relay

Section titled “When to use Manifest alone vs Manifest with Relay”

Use Registry Manifest alone when:

  • You need static files for an open-data portal or a standards body’s discovery index.
  • You want to validate a manifest in CI before any deployment.
  • Your deployment environment cannot run a persistent service.
  • You want interoperability artifacts (DCAT, CPSV-AP, SHACL) without a query API.

Add Registry Relay when:

  • Callers need HTTP row access, aggregates, or evidence verification in addition to metadata.
  • You need per-caller scoping so different systems see different dataset views.
  • You need tamper-evident audit records tied to individual principals and requests.
  • You need metadata publication that responds to caller identity, not static file hosting.

The two are not alternatives for the metadata use case; they are additive. A deployment running Relay will typically also have a static publication pipeline for the same manifest, so the metadata is available even when Relay is down for maintenance.

Where the line falls

Section titled “Where the line falls”

The boundary between what the manifest owns and what the runtime config owns is enforced at startup. A manifest must not contain source paths, table names, database scopes, or backend credentials. Those belong in config.yaml. The boundaries and map page describes the full scope of each project’s responsibilities, including where Registry Platform’s shared primitives fit in.