Keeping Package Code Examples Up to Date

Package maintainers publish code examples in READMEs, migration pages, configuration guides, API documentation, and sample projects. When an API, option, default, or supported setup changes, an old snippet can teach users a call that no longer works or a configuration the package no longer accepts.

Give each example an owner, a version scope, and a check. The version scope says which package release the example describes. An audit means finding the examples, matching them to the changes they describe, and testing or reviewing them before release.

Inventory examples across the repository

Start by searching beyond the main README. Look in docstrings, tutorials, and release notes, as well as notebooks and sample directories. Also check commands that users copy into a terminal. Include configuration fragments and both sides of migration instructions: the old form and the replacement.

Record each example’s location and what it demonstrates, along with the release it describes. Note where its displayed code comes from and how you validate it. A lightweight file or review checklist is enough. This record helps reviewers tell a current API tutorial from a migration page that shows syntax from an older release.

Use package changes to guide the audit. When a pull request changes a public method or option, identify every example that uses that behavior. Do the same when it changes a default or supported setup. Update affected documentation in the same review as the implementation change, following Google’s documentation guidance. Remove or clearly mark examples that no longer describe a supported path rather than leaving them beside current instructions.

Keep migration examples tied to their releases

Migration instructions describe a move between two versions. Label both ends of that move, then check the old and new code against the package behavior for the corresponding releases. If a migration example silently adopts current syntax on both sides, readers upgrading from an older release lose the information they need.

Keep current documentation and historical documentation distinct when your package supports multiple documentation versions. Docusaurus versioning separates current documentation from versioned copies. Django’s upgrade guidance directs readers to release notes and deprecations across the releases they are upgrading through. These approaches help maintainers update the right page while keeping older examples in the syntax for their release.

When a release changes a configuration option, update the migration path and the relevant release notes together. Do the same for changes to a name or default. Preserve older instructions when they still serve users of supported releases, and state which release range each page covers.

Match the check to the example

Run code examples where the documentation tools support it. A doctest is a tool that finds examples in documentation, runs them, and compares the result with expected output. Python’s doctest checks examples in text files and docstrings. Sphinx’s sphinx.ext.doctest runs marked documentation blocks, including setup shared by examples in the same document. Rust’s documentation tests compile and run code in documentation, with options for examples that should only compile or should fail. Go’s testing package distinguishes examples with expected output from examples that are compiled without being executed.

Choose a check that matches what the snippet promises. Compile or run a code example when it presents a working call. Compare output when the output is stable and useful to the reader; avoid depending on incidental formatting. For a command, make prerequisites clear and test it in an environment that matches those instructions. If credentials or an external service prevent an automated check, state the limitation and review the example manually.

Configuration needs its own validation. First check that the snippet parses in its stated format. If the package publishes or uses a schema, validate the structure against it; JSON Schema describes validation as checking data against a schema. Then test the configuration through the package’s own loading path. A configuration file can be valid JSON or YAML and still use an unsupported option or produce the wrong package behavior.

Reduce copied code and run checks in CI

When documentation repeats code maintained elsewhere, include the source file or the relevant part of it if doing so preserves the example’s meaning. Sphinx’s literalinclude displays code from an external file and can select part of it. Inclusion keeps displayed code connected to its source, but it does not prove that the code works. Test that source separately.

If an example needs to stay embedded in explanatory prose, extract or execute it with the documentation toolchain instead. Keep the explanation around it: a runnable snippet still needs to tell readers where it fits and what result to expect.

Run fast documentation checks during pull-request review. Continuous integration, or CI, means automated checks that run when repository changes trigger them. A GitHub Actions workflow can build documentation and run example tests for pull requests. Configure CI to check snippets, configuration, and documentation builds during pull-request review. Run broader sample-project checks when preparing a release. A passing check covers only the code path, dependencies, and environment it tests. Review examples for clarity yourself, and review any examples the tools cannot execute.

Verify the published package page

Review the rendered documentation after publication, not just the source files. For npm packages, the package README guidance says the package page’s README updates when a new version is published. Check that page after the release, especially when install commands or examples changed. Other package registries may handle README publication differently, so verify the behavior of the registry you use.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com