Technical guide library
Detailed interface and ingestion references, each built around a distinct contract, failure mode or decision.
Design a normalized record without erasing provider meaning
Specify a small shared observation contract while retaining the provider fields needed to interpret units, periods, availability and revisions.
Parse numbers only after specifying their units
Handle numeric strings, finite values, percentage scales and magnitude conversions with explicit, testable transformation rules.
Keep zero, null, absence and request failure distinct
Build missingness policies that preserve the difference between unavailable observations, malformed records and failed retrievals.
Version the meaning of a schema, not only its field names
Classify integration changes by their effect on consumers and test migrations against preserved fixtures and semantic expectations.
Separate observation, availability and retrieval clocks
Record what a value describes, what evidence supports its availability, and when the integration actually obtained it.
A date-only availability field cannot answer an intraday question
Use explicit inclusion policies for daily vintage and filing dates without manufacturing release times or claiming unsupported point-in-time precision.
Store revisions as evidence instead of overwriting history
Distinguish changed values, repeated retrievals and conflicting source identities while preserving the version that a research decision used.
Build an as-of join that explains what it can prove
Join observations to decision cutoffs using availability evidence, revision rules and coverage checks rather than observation dates alone.
Prove that pagination finished before publishing a dataset
Reconcile page metadata, expected totals, unique identities and mutable-source behavior instead of equating a successful first response with a complete download.
Retry failures within a bounded, provider-aware budget
Classify transient and permanent failures, respect Retry-After and coordinate request rates without turning a temporary outage into an endless ingestion loop.
Make replay safe and commit checkpoints atomically
Prevent duplicate observations and lost pages by separating fetch attempts, durable records and the checkpoint that declares a run complete.
Distinguish HTTP cache freshness from research-data freshness
Use validators and cache policies correctly while separately measuring retrieval age, observation age and revision coverage.
Integrate the documented FRED V1 observation shape
Map FRED series observations and metadata while distinguishing V1 fixtures, the newer V2 offering, daily vintages and series-specific usage rights.
Integrate World Bank Indicators V2 without inventing release history
Interpret the response envelope, annual periods, country metadata, null observations and retrieval-based availability in a bounded adapter.
Integrate SEC company facts with accounting context intact
Select entity, taxonomy, concept, unit and period explicitly while preserving filing-date precision and accession ambiguity.
Compare provider interfaces by the evidence they actually expose
Use a technical fit matrix for history, identity, timing, errors and rights instead of ranking APIs by a single speed or field-count score.
Keep API credentials out of fixtures, browsers and public receipts
Design a credential boundary that supports reproducible demonstrations and authorized server-side collection without leaking keys through URLs, logs or exports.
Review data rights at the dataset and output level
Distinguish API access, underlying data permissions, attribution and redistribution, while preserving a documented World Bank terms-link gap.
Package an integration result with a reproducible provenance manifest
Connect raw evidence, normalization, selected revisions and published outputs without exposing credentials or mistaking file hashes for truth.
Evaluate an integration with adversarial fixtures and explicit release gates
Create a testable acceptance pack for schemas, time semantics, failure handling, rights and deployment without treating a demo as a production service.