Your API examples compile. Do they actually work?
Run published examples against isolated fixtures and verify their outcomes. Compilation catches only one class of documentation failure.
A reader installs your SDK and copies the first example. It compiles, then fails because the guide uses an old authentication flow. Compilation checked the syntax and types. It did not check whether the published instructions still work against the API.
Run the supported examples in CI with declared dependencies, test credentials and assertions against the results. Include the code readers see, not a separately maintained version that happens to pass.
Test the example readers receive
Keep a canonical example source and include that source in the published page, or extract the published block into a test fixture. Either method needs a check that the tested code and displayed code agree. A test against a different snippet says nothing about the code on the page.
Define the execution context: language version, dependency versions, environment variable names, API version and starting state. Pin dependencies for reproducible CI, and periodically test compatible newer versions separately. Do not quietly replace the advertised package version with a different internal build.
For fragments that intentionally omit setup, document what the surrounding application supplies. Test them inside a minimal harness that provides exactly that setup. Label pseudocode clearly so readers do not mistake it for a runnable program.
Verify the business outcome
Check the resource created by the example as well as its exit code. If an example creates a subscription, independently verify the expected test subscription and its important fields. If it lists resources, supply enough fixtures to check the behavior described in the guide.
Use synthetic data and sandbox credentials with permissions limited to the example. Keep real credentials out of Markdown, build output and stored transcripts. Capture sanitized errors that help an author understand whether a failure came from the example, fixture setup or service availability.
Make cleanup run even when the main assertion fails. Prefer disposable fixture namespaces so parallel CI runs cannot delete each other's resources. If cleanup itself fails, surface that result instead of letting orphaned state accumulate unnoticed.
Separate deterministic checks from live checks
Syntax checks, formatting and static analysis are cheap and useful on every content change. Local contract fixtures can exercise many examples predictably. Live sandbox tests verify service integration, but introduce availability, quota and timing dependencies.
Report a sandbox outage as an environment failure. Decide whether it blocks publication, and avoid an unconditional retry loop that hides it. Bound retries and keep the first failure available for diagnosis.
GitHub Actions' CI documentation explains the workflow mechanisms for running automated builds and tests. The extraction, fixture and outcome checks described here are an authoring approach you can implement with your existing CI system, not a universal built-in feature.
Report the failing example
Report the article, example label, package version and failed assertion. Link the author directly to the displayed code. When an API change invalidates an example, update the contract, code and explanation together.
