Distinguish HTTP cache freshness from research-data freshness

Identify which freshness question is being asked

HTTP caching asks whether a stored response may satisfy a request under the protocol’s rules. Collection freshness asks when the integration last obtained or revalidated a response. Research freshness asks whether the relevant observation and revision are suitable for the current analysis.

RFC 9111 defines caching behavior, including revalidation. Its no-cache directive does not simply mean that storage is forbidden; no-store addresses storage restrictions. An integration should honor the actual response directives rather than treating cache headers as hints that can be ignored whenever a source becomes inconvenient to query.

A synthetic example with three ages

Imagine a synthetic annual observation for 2024 retained in a response retrieved on 2026-04-01 at 09:00 UTC. A successful revalidation occurs at 09:30, and a researcher opens the dataset at 09:40. Time since revalidation is ten minutes; time since the original retrieval is forty minutes; the observation still describes 2024.

A dashboard that displays only 'updated ten minutes ago' can mislead readers into thinking the underlying measurement is recent. Show the observation period, retrieval or validation time and availability basis. If the source provides no observation-specific release history, revalidation does not retroactively establish when the value first became public.

Use validators only when the server supports them

ETag and Last-Modified can support conditional requests where supplied and applicable. A 304 response lets a cache reuse its corresponding stored representation under the protocol rules. It is not a new observation payload and should not be parsed as an empty dataset.

In a synthetic test, retain body B with validator V, send a conditional request and receive 304. The accepted observations remain those from B, while the validation event is recorded separately. If B is missing locally, the integration cannot reconstruct it from a 304 status alone and should recover with an appropriate full retrieval.

Store successful revalidation as its own event. Keeping the original body retrieval timestamp alongside a later validation timestamp prevents an operational status update from rewriting the historical evidence attached to the dataset.

Build cache identity from the representation request

Include every parameter that changes the returned representation: provider, endpoint, series, country, period range, output mode, units and vintage selection. A cache keyed only by endpoint can return one series’ data for another or reuse a current response for a historical-vintage query.

Keep credentials out of public cache keys and logs, while preserving any necessary separation between authorized users or entitlements. A response permitted for one access context may not be safe to share with another. The examples in this publication contain synthetic public fixtures, so they do not demonstrate entitlement-aware production caching.

Make stale fallback an explicit analytical state

If a live refresh fails, serving the last accepted dataset may be appropriate for some read-only uses, but it must remain labeled with its original evidence and age. Do not update retrieved_at to the current time merely because the cached file was opened again. A new local read is not a new upstream retrieval.

Set an analysis-specific staleness policy and test the transition from current to stale to unavailable. A critical input can require withholding a calculation, while an educational chart may retain an older labeled snapshot. These worked examples describe design criteria; the supplied offline adapters do not implement an HTTP cache or establish provider uptime, current economic accuracy or a blanket right to retain and redistribute cached data.

Keep in the decision record

• Keep cache validity, retrieval age and observation age separate.
• A 304 validates a retained representation; it is not an empty data response.
• Never relabel an old snapshot as newly retrieved.

Source notes

1. RFC 9111 — HTTP Caching, June 2022
2. RFC 9110 — HTTP Semantics, validators and conditional requests
3. World Bank — Summary Terms of Use

Primary specifications support the interface descriptions. All downloadable response fixtures are synthetic. Access and rights checks are dated in the source inventory.

Continue the analysis