Package and run a Registry Discovery index

If you run the index that applications search, you maintain one project directory, package each revision of it into a new directory, and restart Discovery on that package. You need the discovery-project directory and the running discovery from Publish and consume a Registry Discovery index, and a second shell at the checkout root for the commands that run beside it.

Install the binaries

Releases from v0.38.0 publish Linux amd64 assets discovery-<tag>-linux-amd64 and discoveryctl-<tag>-linux-amd64, with <tag> from the latest release. Authenticate them through the release’s SHA256SUMS as the verification guide describes, then install them under their plain names:

tag="${TAG:?set TAG to the published release tag}"
mkdir -p ~/.local/bin
for binary in discovery discoveryctl; do
  install -m 0755 "${binary}-${tag}-linux-amd64" "$HOME/.local/bin/${binary}"
done

On another platform, build them from a checkout as the tutorial does. The release also publishes the image ghcr.io/registrystack/discovery:<tag>; deploy it by the digest the release manifest records, not by the tag. It runs discovery as a nonroot user and reads /etc/registry-discovery/runtime.yaml. Mount that file and the package directory read-only; Discovery writes nothing.

Approve HTTPS origins

A provider gives you the HTTPS URL of its public description out of band. Check the project the way production does, without --allow-loopback:

discoveryctl check --project discovery-project

discoveryctl check refused the input.
error[discovery.origins.catalog-url-not-allowed] discovery-project/origins.yaml:5:17 /origins/0/catalogUrl
  a catalogUrl is an https URL with no query, fragment, empty path segment, whitespace, or control character
  next: Write an https catalog URL with no query or fragment, or pass --allow-loopback for an http loopback origin during development.
1 error, 0 warnings in 3 files

In production, write the provider’s URL as catalogUrl, one entry per provider with its own originId. The steps below keep the local origin so that you can package again on your machine.

Add a file under discovery-project/mappings/ for each requirement applications resolve. A mapping may list several alternatives; the types inside one alternative are required together, and the application chooses one alternative. jurisdiction is optional, and a resolve request must match it exactly when it is set.

Package a revision

Stop the running discovery with Ctrl+C. In the second shell, start the publication server again:

python3 products/discovery/tutorial/publication_server.py \
  --descriptions products/discovery/fixtures/descriptions

A package is never replaced in place. In the shell where discovery ran, packaging into the existing directory is refused:

discoveryctl package \
  --project discovery-project \
  --output discovery-project/package \
  --allow-loopback

discovery-project/package already exists; `discoveryctl package` writes a package into a new directory

Package into a new directory, and record your own revision label with it:

discoveryctl package \
  --project discovery-project \
  --output discovery-project/package-2 \
  --allow-loopback \
  --revision 2

packaged packageDigest=sha256:<package-digest> catalogRevision=sha256:7e0799f2c5fd24957e4d1372f757c40abef694110eb70fc260a0abdc312ca963 mappingRevision=sha256:332004ca3920c498539180946e8f2637e9998ba7e49cd98f31e19d6f818857ac

The two revisions are unchanged, because the descriptions and mappings mean the same. The package digest is new: it is the SHA-256 of the package’s SHA256SUMS file, which covers the index, its fetch times, and the REVISION file. A failed run leaves no directory behind. Stop the publication server with Ctrl+C.

Check the index on its own, as Discovery will read it:

discoveryctl check --index discovery-project/package-2/discovery-index.json

valid index origins=1 services=1 mappings=1 catalogRevision=sha256:7e0799f2c5fd24957e4d1372f757c40abef694110eb70fc260a0abdc312ca963 mappingRevision=sha256:332004ca3920c498539180946e8f2637e9998ba7e49cd98f31e19d6f818857ac
0 errors, 0 warnings in 1 file

Pin the package

Pin the package you reviewed, so that Discovery refuses any other. Add the pin to discovery-project/runtime.yaml:

discovery-project/runtime.yaml

package:
  root: ${DISCOVERY_PACKAGE_ROOT}
  expectedDigest: ${DISCOVERY_PACKAGE_DIGEST}

Set both values. In production, copy the digest from the packaged line; here, compute it from the same file (shasum -a 256 on macOS, sha256sum on Linux):

export DISCOVERY_PACKAGE_ROOT="$PWD/discovery-project/package-2"
export DISCOVERY_PACKAGE_DIGEST="sha256:$(shasum -a 256 "$DISCOVERY_PACKAGE_ROOT/SHA256SUMS" | cut -d ' ' -f 1)"

Check the runtime file with the values filled in:

discoveryctl check --runtime-config "$PWD/discovery-project/runtime.yaml" --environment

0 errors, 0 warnings in 1 file

Without --environment, the check reads the file without filling its ${NAME} expressions.

Restart on the new package

Discovery checks the pin before it listens. Started on the old package, it refuses:

DISCOVERY_PACKAGE_ROOT="$PWD/discovery-project/package" \
  discovery --runtime-config "$PWD/discovery-project/runtime.yaml"

discovery: package.expectedDigest is sha256:<pinned-digest> but the package at package.root is sha256:<deployed-digest>; deploy the pinned package or update package.expectedDigest

Start it on the pinned package:

discovery --runtime-config "$PWD/discovery-project/runtime.yaml"

In the second shell:

curl -s http://127.0.0.1:38080/ready

{"status":"ready"}

Before it listens, Discovery recomputes every SHA256SUMS entry and refuses a changed, missing, or extra file by name. It serves that index until it stops; a new revision always means a new package and a restart.

The runtime file is the whole runtime configuration: the listener, the package root and pin, the limits, and the log level. It holds no origin, mapping, credential, or trust setting. Discovery serves plain HTTP without authentication on whatever address listener.bind names, the example being 127.0.0.1:38080, so terminate TLS in front of it. Keep both paths absolute and free of symbolic links.

Maintain the index

Who	Keeps	Acts when	By
Provider	Its public description	Its endpoint, profile, or Evidence types change	Publishing the new bytes at the approved URL
Operator	origins.yaml, mappings, runtime.yaml, the deployed package	An origin or mapping changes, or a provider republishes	Check, package into a new directory, pin, restart
Application	Its saved selections and its own Evidence trust	A saved selection’s service, endpoint, roles, jurisdictions, capability, mapping, or origin changes; a new catalog revision or originContentDigest alone does not	Re-deciding trust, then calling the provider directly

Discovery never trusts a provider for the application: How Registry Discovery works explains that boundary.

Troubleshooting

Symptom	Cause	Fix
discoveryctl check refuses a file	It does not match its closed format	Make the edit each finding names at its line and column
Packaging cannot fetch an origin	The description is unavailable or the URL breaks the fetch policy	Restore the approved publication, then package into a new directory
Startup reports a digest mismatch	The package is incomplete, changed, or not the pinned one	Deploy the complete pinned package, or update package.expectedDigest
Startup says the index predates apiVersion and kind	The package was built by an older discoveryctl	Package it again, then update the pin
Search returns no items	No indexed description matches every filter	Review the indexed descriptions and the identifiers you sent

Check commands

Every file Registry Discovery reads has a command that checks it offline. Each prints the diagnostics, JSON output, and exit codes that Check a file before you deploy it describes.

Format and files	Check command
Discovery approved origins origins.yaml	discoveryctl check --project <project>
Discovery Evidence mapping mappings/*.yaml	discoveryctl check --project <project>
Discovery runtime configuration runtime.yaml	discoveryctl check --runtime-config <file>

Clean up

Stop Discovery with Ctrl+C, then remove the project and its packages:

rm -r discovery-project