Designing Provider APIs with Explicit Boundaries

Provider integrations are easy to start and hard to keep healthy. The first version often mirrors a vendor’s JSON, exposes the HTTP client directly, and gets a feature into production quickly. The cost arrives later: every wire-level rename becomes a breaking change, every provider quirk leaks into application code, and tests become coupled to one vendor’s vocabulary.

The better boundary is not a large abstraction that pretends providers are identical. It is a small, owned API that describes what the application means while keeping provider-specific work behind an adapter.

Start with the application decision

Before designing a type, write down the decision the caller needs to make. A text-generation caller may need a request with messages, a model identifier, a token limit, and a stream of output. It usually does not need to know whether a provider calls the limit max_tokens, max_completion_tokens, or something else.

That distinction suggests three layers:

1. Semantic types owned by the library, such as Message, Role, Model, and CompletionRequest.
2. A capability boundary that describes operations the library actually supports.
3. Private provider wiring that translates those types into URLs, headers, JSON, retries, and response envelopes.

The public layer should be boring to use. If the caller needs a provider-specific escape hatch, make that an explicit extension point rather than quietly adding vendor fields to every common request.

Keep the common denominator honest

An abstraction becomes painful when it promises features that the providers cannot share reliably. Streaming, tool calls, usage accounting, and structured output all have different failure modes. Model those capabilities deliberately.

One useful approach is to expose a capability query or separate operation traits. A provider can support text generation without claiming to support tool calls. The caller can then handle an unsupported capability as a normal, typed error rather than discovering it through a remote 400 response.

This is also where explicit limits matter. Request size, maximum output, timeout policy, and stream buffering should have documented defaults and bounded behavior. A limit is part of the API contract; leaving it implicit only moves the decision into a proxy, runtime, or vendor dashboard.

Treat translation as a compatibility firewall

Provider adapters should translate at the edge. Parse and validate a response before returning it, preserve useful provider error context, and normalize only what the semantic API can represent without lying.

Do not discard unknown fields merely because the common type has no use for them. Keep wire models private and version them with the adapter. This lets a provider add fields without forcing a public API release, while still allowing an intentional escape hatch later.

The same rule applies to authentication and transport. A library can accept a credential source and a transport interface without making its public API depend on a particular async runtime, HTTP client, or environment-variable convention. Runtime neutrality is a boundary decision, not a promise that every implementation detail is portable.

Design errors before happy paths

An integration is defined as much by its failures as by its successful response. Give callers enough information to distinguish invalid input, authentication failure, rate limiting, provider unavailability, decoding failure, and cancellation. Preserve a request identifier when the provider supplies one, but do not turn it into the library’s identity model.

Retries belong near the transport boundary and must be conservative. A retry policy should understand idempotency, backoff, deadlines, and cancellation. It should not silently replay an operation merely because a connection closed after bytes were sent.

Tests should exercise the semantic contract and the translation separately. Table-driven tests can prove that a request maps to the expected wire shape; contract tests can prove that each adapter returns the same meaningful result and error categories. This separation keeps a provider change from becoming a rewrite of application tests.

Make evolution an explicit feature

Public APIs are expensive to change, so prefer additive evolution: optional capabilities, new constructors, and new result fields with clear defaults. Avoid exporting generated wire types or re-exporting a vendor SDK as if it were your own. Generated code is useful inside the boundary; it is rarely the right vocabulary for users.

The goal is not to erase provider differences. The goal is to put those differences where they can be named, tested, and changed without surprising every caller. A good provider API gives an application a stable set of decisions while leaving the integration free to evolve behind it.