Skip to content

[Advanced] Add lightweight smoke tests for generated docs example scripts #262

Description

@rodrigo-arenas

Summary

Add lightweight smoke tests for a small subset of generated docs example scripts.

The docs examples are valuable, but it is easy for imports or basic API usage to drift over time. A small smoke-test command would catch simple breakages without running every heavy example.

Why this helps

Examples are often the first code users copy. Smoke tests give maintainers confidence that selected docs examples still import, build their estimators, and run with deterministic small settings.

Suggested implementation

Start by reading these files:

  • docs-vitepress/examples-src/
    • Look at the example source files and choose a few low-cost examples.
  • docs-vitepress/examples-src/README.md
    • Check how examples are generated.
  • docs-vitepress/scripts/
    • Follow the style of existing docs scripts.

Suggested approach:

  • Add a small smoke-test entry point for docs examples.
  • Start with only 2 or 3 low-cost examples.
  • Avoid heavyweight optional dependency examples such as XGBoost, LightGBM, and CatBoost for the first version.
  • Use deterministic seeds and very small population_size / generations settings where examples run optimization.
  • The smoke test should fail with a clear message showing which example failed.

Examples that may be good starting candidates:

  • basic usage
  • feature selection tutorial
  • plotting gallery setup, if it can run cheaply

The final selection can be adjusted after checking runtime.

Tests to add

Add or document validation for:

  1. The smoke-test command runs successfully for the selected examples.
  2. A failing example reports the example name or path.
  3. Optional dependency examples are skipped or excluded intentionally.

Validation

Run the new smoke command, for example:

python docs-vitepress/scripts/<new-smoke-script>.py

Also run any focused tests added for helper code.

Notes for contributors

This is an advanced issue because it touches docs tooling and example runtime behavior. Keep the first version intentionally small so it adds value without making CI slow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementModifications to an existing featurehelp wantedExtra attention is needed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions