How to test an SDK: unit, contract and release tests

How do you test an SDK?

Start with a fake transport so each method can be exercised quickly and deterministically. Check request construction, errors, retries and pagination. Add contract tests against OpenAPI, run the snippets people copy from the README, then test the built package in clean environments. Reserve a smaller sandbox suite for behavior that mocks cannot reproduce.

The levels serve different purposes. Fast local tests protect everyday changes; sandbox tests confirm your assumptions about the real service.

The kinds of SDK test

Do not let one passing sandbox request stand in for all of these. It is entirely possible to reach production successfully while shipping a package with broken exports for another runtime.

Test against a mock transport

I ran a local check with HTTPX's MockTransport, which intercepts requests inside the process. The client performed a GET /v1/orders; the handler recorded the method, URL and Authorization value, then returned a small JSON response without contacting any server. The assertion saw exactly GET, https://api.example.test/v1/orders and Bearer test.

import httpx

seen = []
def reply(request: httpx.Request) -> httpx.Response:
    seen.append((request.method, str(request.url), request.headers['authorization']))
    return httpx.Response(200, json={'data': [1, 2]})

with httpx.Client(base_url='https://api.example.test',
                  transport=httpx.MockTransport(reply)) as client:
    assert client.get('/v1/orders', headers={
        'Authorization': 'Bearer test'
    }).json()['data'] == [1, 2]

assert seen == [('GET', 'https://api.example.test/v1/orders', 'Bearer test')]

That tiny assertion catches the class of mistakes a result-only test misses. In an SDK, route all methods through the injected client so the same technique covers the entire surface.

Test errors, retries and pagination

Exercise a 400, 401, 404, 429, 500, a non-JSON error body, and a connection failure. Check the type and fields of each exception. A 429 should expose retry timing; an unknown order should preserve the request ID; a timeout should not be misreported as a definite server rejection.

For retries, return 503 once and 201 next. Assert that exactly two requests were sent and that a write reused the same idempotency key. For pagination, return two pages and confirm the iterator yields every result with no extra fetch. I separately checked the retry behavior in a small Node mock: two attempts carried the same key.

Contract-test against your OpenAPI spec

Mock tests prove what the SDK does; contract tests establish what it should do. Compare the emitted path, method, parameters and body to the operation's schema. Validate successful and error responses against the documented shapes. This is particularly useful for generated clients, where a missing $ref or an incorrect optional field can spread into many methods at once.

Remember that a spec may be wrong too. Contract tests should complement a small set of calls against a known test environment.

Check that your tests catch bugs

Deliberately remove an Authorization header or change a request path in a local branch. A good suite should fail immediately. Also try changing an SDK method name without updating a README snippet: your documentation test should catch it. Mutation testing need not be elaborate; one controlled regression often reveals whether assertions are checking behavior or merely checking that code runs.

Run the code in your README

Extract copyable code blocks into testable examples, provide fake credentials and point them at a mock server. Type-check the snippets for TypeScript and Python where possible. Do the same for quickstarts on the docs site. A published example is part of your public interface, so a failing example deserves the same attention as a failing method.

Test the package users install

For npm, pack the distribution and install the tarball in an empty project. For Python, build a wheel and source distribution, then install the wheel in a clean virtual environment. Import the advertised entry point, inspect exported types and run the smallest example. This catches missing dist files, incorrect exports, omitted Python modules and accidental dependencies on the source repository.

Test against a real sandbox

Use dedicated test credentials and data. Confirm one successful read, one write that can be safely reversed, one permission error and any special behavior such as file upload or webhooks. Keep these tests bounded and avoid running destructive operations against shared customer accounts.

Common SDK testing mistakes

The most common is asserting only on returned data. Others include sleeping through real backoff delays in unit tests, testing unreleased source instead of the package artifact, using production credentials in CI and treating a regenerated snapshot as automatically approved. A snapshot should require human review when the public tool or method surface changes.

SDK testing checklist

Ask whether every operation has a request-shape test, every error has an assertion, and every retry path is safe. Confirm the full sample suite runs on supported runtimes and that the registry artifact is installable. Check one real sandbox path before promoting a release.

Testing SDKs with Octri

When SDKs come from a shared spec, generated contract tests can protect the mapping from operations to language methods. Octri's generated clients and test surfaces can be part of this pipeline, but the team should still test idiomatic behavior, package installation and real integration paths.