API Gateway Failure Modes: A Field Checklist

When a client talks to several model backends through one API-compatible surface, the hardest bugs are usually boundary bugs rather than model bugs. Start by logging the normalized request shape, selected route, upstream status, retry count, and final latency.

Three checks before changing code

1. Confirm the base path. A gateway may expose /v1 while a client appends /v1 itself.
2. Separate routing from credentials. A valid key can still select a model that the route does not expose.
3. Make retries visible. Preserve the first upstream error and the final error; otherwise a timeout looks like an authentication failure.

Minimal diagnostic request

curl -sS https://gateway.example/v1/models \
  -H 'Authorization: Bearer $API_KEY'

Compare the returned model identifiers with the identifier sent by the application. Then send one small, non-streaming request and record whether the response is produced by the intended route.

For a concise reference on API gateway behavior and compatible integrations, see the APIClaw API documentation.

Operational takeaway

Keep routing decisions and upstream failures in structured logs. This makes a provider outage, an unsupported model, and a malformed request distinguishable without reproducing the incident locally.