Make research API errors machine-readable without hiding failed data

What should an investment data API return on failure?

A caller needs to know whether its request succeeded before interpreting the body as observations. A JSON response containing an error message is not a successful dataset merely because JSON parsing worked. Start with the actual protocol status and expected representation type, then validate the documented success or error contract. Preserve the distinction in logs, exports and downstream eligibility checks.

An empty successful collection means the accepted query found no records within its declared scope. A failed request means that question was not answered. These conditions can require opposite actions: accepting a zero-row population might be valid, while publishing after a failed refresh may be prohibited. Avoid a generic helper that converts both outcomes into the same empty array.

Use stable problem identity instead of message parsing

RFC 9457 defines problem details for HTTP APIs, including type, title, status, detail and instance members. A client should use the problem type to identify the documented condition rather than searching human prose for words such as quota or missing. Human descriptions can change for clarity or language without changing the underlying error's meaning or required response.

The response status and any status member should agree when the server generates them. Preserve both if a received response is inconsistent, because an intermediary may have altered one layer. Treat the mismatch as diagnostic evidence, not permission to choose whichever value produces a successful dashboard. Unrecognized extensions must not cause a consumer to invent undocumented recovery behavior.

Inspect a synthetic invalid-query contract

Consider a fictional request whose cutoff parameter contains an invalid calendar date. The illustrative response uses HTTP 400, content type application/problem+json, type https://example.invalid/problems/invalid-cutoff, and an errors extension naming field cutoff with code invalid-calendar-date. The reserved example address identifies a teaching contract, not an operational endpoint. No real provider response or capability is asserted here.

The caller can associate the failure with its request identifier, display a safe correction hint and stop that dataset generation. It must not insert a null observation under the invalid date. A field-specific extension makes the failure inspectable without forcing software to interpret the prose explanation. Its schema and allowed codes still need documentation; the standard does not define this invented extension.

Classify recovery separately from explanation

An invalid parameter generally requires a changed request, not repeated identical traffic. A temporary service failure may justify a bounded retry when the provider's contract permits one. An authorization failure requires inspection of access context rather than replacing the credential in a public browser example. Define recovery classes explicitly; do not derive a universal retry policy from every error carrying the same broad status.

Keep retry budgets, waiting instructions and terminal states separate from user-facing text. Unknown error types should enter a conservative documented path and retain safe evidence for review. A human-readable detail field can help an operator, but it should not become an executable instruction or an unbounded delay value. The caller remains responsible for its own limits and validation.

Keep partial batches and unsafe diagnostics visible

If a batch contains valid results for some requested identities and failures for others, define whether the overall operation is atomic or explicitly partial. A partial-success envelope should list requested, accepted and failed identities and prohibit a complete-population label. Retaining successful rows can aid investigation without granting them automatic eligibility for a calculation that requires the full request set.

Do not expose tokens, full private query strings, database statements or stack traces in public errors. Use a safe correlation identifier that an authorized reviewer can resolve through protected evidence. RFC 9457's security discussion addresses excessive detail. A helpful message names the rejected field and permitted correction while keeping system internals and confidential research requirements out of reusable fixtures.

Which failure fixtures belong in acceptance?

Test a proxy returning HTML, malformed problem JSON, an unknown problem type, a status/body mismatch and a success body missing required collection metadata. Also test a valid zero-row result. These cases reveal whether the consumer genuinely separates data from delivery outcome or merely catches exceptions and manufactures an empty collection. The latter can silently alter a research universe.

Ask whether each failure prevents publication, permits a labeled stale view or requires manual review. Verify that exported evidence retains that disposition and never presents failure as an observed zero. This guide specifies educational interface behavior; it does not supply a production error service, an access entitlement or a guarantee that an upstream source implements problem details.

Keep in the decision record

• Parsing JSON is not evidence of successful retrieval.
• Problem identity should be stable even when human wording changes.
• A partial batch must retain its failed identities and incomplete scope.

Source notes

1. RFC 9457 - Problem Details for HTTP APIs; public full text reviewed
2. RFC 9110 - HTTP Semantics, status codes and response representation
3. OpenAPI 3.1.1 - response and media-type definitions; public full text reviewed

Primary specifications support the interface descriptions. All downloadable response fixtures are synthetic. Access and rights checks are dated in the source inventory.

Continue the analysis