Design a normalized record without erasing provider meaning

Separate common fields from source evidence

The fixture workbench uses source, series, observation_date, observation_period, period_kind, value, unit, available_date, availability_basis, retrieved_at, revision_key and source_record. These fields distinguish an observation’s subject, numerical representation, period convention and availability evidence. In these adapters, source_record is a provenance locator, not an embedded raw record; preserve the original fixture separately to retain provider-specific distinctions.

JSON Schema provides a vocabulary for declaring object structure and required properties. Use that structural contract to reject malformed normalized rows, then apply semantic checks separately. A schema can establish that unit is a string; it cannot establish that a particular source value actually represents dollars rather than millions of dollars without a documented mapping.

Map a synthetic observation explicitly

Consider a synthetic FRED-shaped item with date 2026-01-01, value string 123.4 and realtime_start 2026-02-10. The adapter may produce source fred, observation_date 2026-01-01, numeric value 123.4, available_date 2026-02-10 and availability_basis fred_realtime_date. The unit must come from selected series metadata or an explicit fixture assumption, not from the number itself.

Retain the raw item and the request’s relevant parameters. The real-time date has daily precision; it must not become an invented 08:30 release timestamp. This invented item demonstrates the mapping only. It is not an observed economic release and does not imply that any real series with these values exists.

Give identity more than a display label

A useful series identity includes the provider namespace and the economic or accounting concept. For a company fact it also needs the entity, taxonomy, tag and unit selection. For a country indicator it needs the country and indicator code. A human-readable title can change without changing identity, while identical titles can describe different series.

Imagine two synthetic records both labeled Revenue with value 100. One is USD millions for a full year; the other is EUR thousands for a quarter. A merge keyed only by label and date would falsely combine them. Preserve the distinctions in identity and metadata before any cross-provider join or display formatting occurs.

Normalize periods without creating timestamps

The World Bank annual fixture uses YYYY-12-31 as a documented period-key convention, while preserving observation_period as the original year and period_kind as annual. December 31 is not a claim that a measurement occurred or was released at that instant. Consumers must read the period metadata when comparing it with monthly or daily observations.

The SEC instant fixture uses the fact’s end date and marks its period kind accordingly. A duration fact would additionally need its start date and context; the present instant mapping does not validate all possible accounting facts. A general interface should reject or explicitly route unsupported period kinds instead of squeezing them into the same date column.

Define a loss budget for normalization

List which source distinctions must survive and which can be intentionally omitted from a specific analysis. For this workbench, availability basis and raw identity evidence are essential. A display label or provider-specific formatting detail may be secondary, but losing it should still be a conscious choice rather than an accidental parser side effect.

Test normalization with records that differ in exactly one meaningful field. Different units, accessions or vintages should not collapse merely because their numerical values match. A successful normalized schema supports reproducible processing; it does not certify comparability, data rights, economic accuracy or point-in-time completeness. Those remain separate questions with their own evidence.

Keep in the decision record

• Preserve provider identity and raw evidence alongside common fields.
• Treat period keys and availability dates as distinct concepts.
• Validate meaning separately from JSON structure.

Source notes

1. JSON Schema — Object reference, Draft 2020-12 context
2. FRED API V1 — Series observations
3. SEC — EDGAR Application Programming Interfaces, updated 8 April 2025

Primary specifications support the interface descriptions. All downloadable response fixtures are synthetic. Access and rights checks are dated in the source inventory.

Continue the analysis