Two pages are one batch only after the contract reconciles

Declare the envelope and row identities

The hypothetical World Bank-shaped example contains two response envelopes. Each holds a metadata object followed by its rows. Page one identifies page 1 and page two identifies page 2; both declare pages 2. Retain per_page, total and lastupdated as metadata, together with the recorded retrieval timestamp 2024-03-15T12:00:00Z. The fictional indicator is SYNTHETIC.DEMO and fictional country identity is XZZ.

Treat the requested indicator, country, date range and format as part of the batch context. Page numbers alone cannot prove that responses belong to the same query. A saved file named page-two is insufficient if its internal metadata says page one or a different indicator appears in its rows. The validation contract should inspect the payload, not infer successful retrieval from a filename.

Reconcile before making data visible downstream

Require the expected page identities without gaps or repeated page numbers. Check that pages, per_page, total and dataset-update metadata agree across the supplied responses, and compare the combined raw row count with the declared total. A shorter final page can be legitimate; a missing entire page cannot be repaired by assuming its records are empty. A successful transport status does not establish any of these properties.

Next inspect record identities across page boundaries. Retain an exact duplicate long enough to report it, and surface different values attached to the same logical observation rather than allowing a dictionary assignment to overwrite one. A batch can have a correct total row count and still contain duplicated observations or conflicting data. Completeness of envelopes and consistency of records are separate checks.

Avoid manufacturing historical availability

Annual periods are retained as observation_period, such as 2023, with period_kind annual. The normalized observation_date is 2023-12-31 solely as a sortable period key. It does not mean the annual observation was measured on that day or published before the year ended. The original period remains available so a consumer cannot mistake the sorting convention for source precision.

The availability_basis is retrieval_time. Dataset lastupdated describes the dataset metadata, not a verified publication timestamp for every row. Keep the actual recorded retrieval timestamp and do not backfill an earlier available_date from the observation year. A historical cutoff before this retrieval evidence cannot establish that the same values were accessible then, even when their observation periods are much older.

Preserve a reproducible integration boundary

The World Bank documentation defines page and per_page request controls. This local example adds an explicit acceptance contract for the selected response shape. It is not a promise that every provider response mode has been implemented, nor a substitute for a fresh integration test if the upstream interface changes. Empty results, error payloads and metadata changes need tests separate from the happy two-page path.

Run the fixture without a network connection, examine the normalized rows and inspect all diagnostics. Save the envelope metadata with the result. In a real ingestion service, make the validated batch and its checkpoint visible together only after every required page passes. The browser lab demonstrates the contract; it does not operate a scheduler, maintain a durable production database or guarantee that a live source remained unchanged between page requests. A changing dataset may require restarting a snapshot even when each page is syntactically valid.

Keep in the decision record

• Validate envelopes and record identities independently.
• An annual sorting key is not a release date.
• Keep lastupdated as dataset metadata and retrieval evidence as retrieval evidence.

Source notes

1. World Bank: API Basic Call Structures

Primary specifications support the interface descriptions. All downloadable response fixtures are synthetic. Access and rights checks are dated in the source inventory.