Store revisions as evidence instead of overwriting history

Separate observation identity from version identity

An observation identity describes the thing measured: provider, series, unit and period, with entity or geography where needed. A revision identity distinguishes versions of that observation. The normalized revision_key supports this second purpose, but its construction must follow a documented provider mapping.

A retrieval identifier serves another purpose: it records a particular acquisition event. Repeatedly downloading an unchanged value should not create multiple economic observations. Conversely, replacing an earlier value with a later one under the same period key destroys the evidence needed to reconstruct what a prior analysis actually used.

A synthetic two-vintage example

Suppose a synthetic January measure is 100 in a February 10 vintage and 101.2 in a March 10 vintage. Both records describe January, but they support different historical questions. A current chart may choose 101.2; a February snapshot may need 100. Preserve both and the policy used to select between them.

FRED’s real-time-period documentation supports date-bounded views of information and revisions. The fixture mapping retains that daily basis. It does not manufacture a precise publication time or guarantee that every historical revision for every series has been retrieved. The chosen request scope and endpoint output mode remain part of the evidence.

Do not infer revisions from file order

An API may return several records for the same period. Their array order is not a universal version ordering contract. Sort or select only using documented fields and declared tie rules. If two candidates have the same availability date and conflicting values, retain the conflict instead of assuming the last array item is authoritative.

For SEC-shaped synthetic facts, different accession numbers can represent different filings or contexts. A filing date alone may not resolve their order or comparability. The adapter stores accession evidence in source_record; retain the raw payload separately for additional context. An amendment or repeated comparative disclosure deserves explicit interpretation rather than an automatic replace-latest operation.

Use content comparison carefully

A content hash can reveal that the retained bytes changed, but it does not explain why. Reordered JSON properties can change a byte hash without changing the observation. A semantic digest based on a canonical selection of meaningful fields can help distinguish content changes from formatting, provided the canonicalization rule is versioned.

In a synthetic replay, retrieve value 100 twice with different retrieval timestamps. The observation revision should remain the same if its source revision evidence is unchanged. Then retrieve 101.2 with a new vintage marker: the history should gain a new version. Finally, retrieve conflicting content under the same supposedly immutable key and require a visible conflict diagnostic.

Preserve selection and correction history

A research artifact should record which revision keys it consumed. If an adapter bug later changes normalization, replay the raw evidence under a new mapping version and label the corrected result. Do not present the repaired output as the exact dataset originally available to the old analysis.

A complete revision store also needs a policy for deletions, withdrawn records and inaccessible history. Where only present-day values were collected, acknowledge that earlier versions are missing. This guide’s invented sequence demonstrates storage and selection behavior; it does not create a historical release archive or prove that a strategy’s information set has been fully reconstructed.

Keep in the decision record

• Observation identity, revision identity and retrieval identity are different.
• Keep same-day conflicts and accession ambiguity visible.
• Record the exact selected versions in research outputs.

Source notes

1. FRED API V1 — Real-Time Periods
2. FRED API V1 — Series vintage dates
3. SEC — Accessing EDGAR Data, correction and deletion discussion

Primary specifications support the interface descriptions. All downloadable response fixtures are synthetic. Access and rights checks are dated in the source inventory.

Continue the analysis