Package an integration result with a reproducible provenance manifest

Represent inputs, activities and outputs

W3C’s PROV-O recommendation distinguishes entities, activities and responsible agents. That is a useful conceptual reference for a compact integration manifest: raw response artifacts are inputs, normalization is an activity, and accepted normalized records are outputs. The publication’s JSON manifests need not claim formal PROV-O conformance to preserve that chain.

Record source interface, safe request parameters, retrieval time, raw artifact digest, schema version, mapping version and output digest. Include record counts and validation status. Keep the credential itself out of the manifest; a safe internal reference can identify the authorized configuration without revealing its secret value.

Trace a synthetic result end to end

Suppose an invented payload contains two observations, one numeric and one null. Normalization produces two accepted records and one observed numeric value. A manifest should reconcile raw count 2, normalized count 2 and non-null count 1. A chart showing one point should still link to the manifest that explains the missing value.

Now run a derived calculation that averages only observed values. Record the calculation version, missingness policy and selected revision keys. A later reviewer can then distinguish a changed source payload from a changed formula or a changed selection rule. A single final CSV without this chain cannot reveal which layer caused a difference.

Use hashes with a declared representation

A byte digest identifies the exact stored bytes. JSON property order and whitespace can change that digest without changing the meaning. If semantic comparison is needed, define a canonicalization rule and preserve its version. RFC 8785 specifies one JSON canonicalization scheme, but an application should not claim to implement it unless it actually follows and tests that specification.

For a synthetic test, reorder two JSON properties and verify that byte hashes differ while the normalized record remains equivalent. Then change value 10 to 11 and verify that the semantic comparison detects the difference. This separates formatting changes from analytical changes without implying that either hash establishes authenticity.

Keep unavailable evidence visible

A manifest can record that a source link was retrieved, redirected, blocked or unavailable. These statuses are evidence, not inconveniences to erase. The World Bank full dataset-terms redirect gap is an example of why a citation alone should not be counted as a successful document review.

If a raw response cannot be redistributed, record its identifier and permitted retention location without attaching it to a public download. If the response contains sensitive material, separate protected evidence from the public manifest. Reproducibility should be pursued within actual rights and access boundaries rather than by copying everything into a public archive.

Tie published claims to the accepted generation

A release note should identify the exact artifact generation used for the displayed results. If code or fixtures change, regenerate the output and manifest together. A screenshot from one version and a download from another can produce a misleading appearance of verification even when each file is individually valid.

The fixture lab’s manifests support reproducibility of synthetic examples and mapping behavior. They do not document a live subscription, cloud ingestion service or historical provider archive unless such operations are separately performed and evidenced. Preserve the distinction so readers can inspect a working technical sample without mistaking its provenance record for an operational capability that has not been established.

Keep in the decision record

• Record the transformation chain, not just the final file.
• Distinguish byte identity, semantic equivalence and source truth.
• Keep rights restrictions and failed evidence retrieval in the manifest.

Source notes

1. W3C — PROV-O Recommendation, 30 April 2013
2. RFC 8785 — JSON Canonicalization Scheme, June 2020
3. World Bank — General Terms and Conditions

Primary specifications support the interface descriptions. All downloadable response fixtures are synthetic. Access and rights checks are dated in the source inventory.

Continue the analysis