Specify filters and sort order before trusting an API query
Make query boundaries, parameter serialization, defaults and stable ordering explicit so a successful response answers the intended research question.
A reproducible research query needs declared filter semantics, boundary inclusivity, default behavior and deterministic ordering. Preserve the interpreted query beside its response; a valid URL and a successful status do not prove that the server understood the population you intended.
Why can a successful query return the wrong sample?
A request may be syntactically valid while selecting the wrong population. A comma-separated identifier list can be interpreted as one literal identifier; an omitted end date can mean latest available; and a date boundary can be inclusive or exclusive. None of these ambiguities necessarily causes a server error. The response can therefore look complete while answering a different research question.
Write the intended query in a compact human-readable statement before encoding it. Name the identity universe, economic periods, availability cutoff, units and ordering. Separate observation-period filtering from version-availability filtering. Selecting records dated before a decision does not by itself exclude later revisions of those records. The server must document which dimension each parameter controls.
Define parameter serialization and defaults
OpenAPI's parameter definitions describe locations, schemas and serialization choices. RFC 3986 defines URI syntax and percent encoding, but neither tells you a dataset's economic interpretation. Review the actual operation contract. Specify whether a list uses repeated parameters or a documented delimiter, how an empty list behaves, and whether an omitted value differs from an explicitly empty value.
Treat unsupported parameters as an acceptance question. Some interfaces reject them; others ignore them. A misspelled cutoff that is silently ignored is particularly dangerous because it can expose later information without changing the success status. A consumer can validate its own allowed names and preserve a normalized query manifest, but it must not claim that local validation proves server enforcement.
Check a synthetic half-open interval
Suppose an invented endpoint supports observation_date greater than or equal to 2024-01-01 and strictly less than 2024-04-01. Candidate records are dated January 1, March 31 and April 1. The half-open rule includes the first two and excludes April 1, giving two selected rows. This is a calendar contract example, not a historical observation sample or investment result.
If the next request covers April 1 through, but not including, July 1, the shared boundary belongs only to the second request. Inclusive endpoints on both requests would duplicate April 1 unless deduplicated by stable identity. Record the boundary convention explicitly rather than relying on a display label such as first quarter, which does not define timestamp or date interpretation.
| Synthetic candidate date | Rule: 2024-01-01 <= date < 2024-04-01 | Selected |
|---|---|---|
| 2024-01-01 | At included lower boundary | Yes |
| 2024-03-31 | Inside interval | Yes |
| 2024-04-01 | At excluded upper boundary | No |
| Selected count | 2 of 3 candidates | 2 |
Make ordering total enough for pagination
A sort by observation date alone leaves ties when multiple series or versions share that date. Add documented tie-breakers such as series identity and revision identity, while respecting any ambiguity in source chronology. A deterministic order need not assert economic precedence: an identifier can break a display tie without proving that one revision became available before another.
Cursor pagination should state which query and ordering the cursor belongs to and whether the underlying population is stable. Reusing a cursor after changing filters can skip or repeat rows. Offset pagination over a changing dataset has its own movement risks. Do not solve these problems by simply sorting the final retrieved subset; absent records cannot be recovered by rearranging present ones.
Separate query equivalence from byte equivalence
Two differently encoded requests can be semantically equivalent, while two nearly identical strings can select different populations. Retain the exact safe request representation and an independently normalized interpretation. Do not sort repeated parameters or collapse duplicates unless the operation defines their meaning as order-insensitive. A generic URL canonicalizer can otherwise change an intentional sequence or conceal an invalid duplicate.
Cache identity and provenance should include parameters that affect meaning, including defaults resolved at request time. A request for latest cannot be reproduced months later from the word latest alone. Preserve the selected generation or response evidence. Credentials must stay outside public manifests even when authorization affects the accessible population; a safe entitlement reference can document that boundary.
What should a query acceptance test challenge?
Ask what happens at both boundaries, with an empty identifier list, an unknown identifier, duplicate parameters, an unsupported unit and a misspelled field. Include two records tied on the primary sort key and a cursor reused after a filter change. Each expected outcome should follow the written contract rather than whatever a convenient client library happens to return.
Reject a research-ready claim if the interface cannot explain its effective defaults, population boundaries or incomplete-query behavior. A narrow supported query is more inspectable than an undocumented flexible one. The teaching contract here proposes questions and fixtures; it does not assert that any provider offers these exact parameters, preserves historical snapshots or grants a particular usage right.
Keep in the decision record
- Preserve the interpreted query as well as its safe encoded form.
- Observation dates and availability cutoffs filter different dimensions.
- Stable sorting cannot repair missing rows from an unstable pagination boundary.
Source notes
- OpenAPI Specification 3.1.1 - Parameter Object and serialization; public documentation reviewed
- RFC 3986 - URI syntax and percent encoding; public full text reviewed
Primary specifications support the interface descriptions. All downloadable response fixtures are synthetic. Access and rights checks are dated in the source inventory.
Continue the analysis
Build an as-of join that explains what it can prove
Join observations to decision cutoffs using availability evidence, revision rules and coverage checks rather than observation dates alone.
Prove that pagination finished before publishing a dataset
Reconcile page metadata, expected totals, unique identities and mutable-source behavior instead of equating a successful first response with a complete download.
Distinguish HTTP cache freshness from research-data freshness
Use validators and cache policies correctly while separately measuring retrieval age, observation age and revision coverage.
Make research API errors machine-readable without hiding failed data
Separate request failure, invalid inputs and partial results using explicit error contracts, safe diagnostics and inspectable caller behavior.
Version 1 · Published 16 October 2026. Inspect the method · Open the contract lab