Skip to content

DOC: freeze old whatsnew notes (GH#6856) - #67290

Open
jbrockmendel wants to merge 4 commits into
pandas-dev:mainfrom
jbrockmendel:bug-6856
Open

DOC: freeze old whatsnew notes (GH#6856)#67290
jbrockmendel wants to merge 4 commits into
pandas-dev:mainfrom
jbrockmendel:bug-6856

Conversation

@jbrockmendel

@jbrockmendel jbrockmendel commented Aug 29, 2026

Copy link
Copy Markdown
Member

closes #6856

The .. ipython:: blocks in the whatsnew files are re-executed on every doc build against whatever pandas is current. That is why they end up illustrating behavior other than the release they describe, and why they periodically break the warnings-as-errors doc build when a deprecation they demonstrate is enforced — five blocks in v0.18.0 are currently held up by :okwarning: on the inplace deprecation in eval/query alone.

This rewrites those directives as static .. code-block:: pycon sessions across all 45 files that had executable blocks. 799 executable blocks and 49 :okexcept:/:okwarning: markers go to zero.

The conversion was mechanical, but the script that did it is deliberately not part of this PR — it is a one-shot tool that shells out to pixi exec for pandas 0.17.1–2.x under CPython 3.5/3.6, so committing it would leave unmaintainable tooling in scripts/ and 30 tests running in CI for something nobody will run again. It stays available in this PR's history: freeze_whatsnew.py and its tests, at 82d9c06.

To keep the directory frozen going forward, a no-executable-ipython-in-whatsnew pre-commit hook rejects any .. ipython directive under doc/source/whatsnew/. It passes on the tree as of this PR, including the in-development v3.1.0.

Picking up the two requests from the dev call on the previous attempt (GH-64742): >>> prompts rather than In [n]:/Out[n]:, and output produced by the then-current pandas. The second turned out to be more tractable than that discussion concluded — pixi exec reaches back to 0.17.1, not 2.2. Getting era-faithful output needs three things from the release, not just its pandas:

  • the source, read back with git show <tag>:<file> — these files have been edited for a decade to keep executing, so v2.0.0 says freq='ME' (a 2.2 spelling) and v0.18.0 passes path_or_buf=/header= to Series.to_csv, neither of which existed. Rendering today's text under 2.0.3 gave 15 tracebacks where 4 were intended.
  • the doc header — until 0.24 the whatsnew page did from pandas import *, which is why the released text says Timestamp where today's says pd.Timestamp.
  • doc/data, extracted from the tag, so read_csv('data/baseball.csv') reads the era's file.

conda-forge has no pandas below 0.17.1, so the 16 pre-0.17 files are rendered against the current environment instead (--current). Those pages stop breaking the build but are not historically accurate — v0.13.1 now shows RangeIndex and datetime64[us] where 0.13.1 printed Int64Index and no us. Deliberate; happy to revisit if reviewers would rather see a note on those 16.

Seven blocks render as input with no output, because their output was never reproducible: a Timestamp.now() wall clock, three id() calls used as labels, and a Styler whose repr is a memory address. The script detects these by running each file twice from scratch and refusing anything that disagrees with itself, and pins PYTHONHASHSEED so dict ordering under the CPython 3.5/3.6 interpreters those old releases run on is stable. Two independent full runs produce byte-identical output.

Verified: 45/45 pages build (43 clean; the 2 that warn, unknown document: 'v1.2.1' and '/user_guide/io', warn identically on the unmodified files — single-doc mode excluding the referenced pages), pre-commit and mypy clean.

One pre-existing bug is fixed rather than re-rendered: v0.23.0.rst wrote .. ipython without the ::, so docutils treated it as a comment and its sort=False example rendered as nothing. The new hook matches that missing-colons spelling too, so it had to be resolved here; the block is now frozen like its two siblings.

AI disclosure: drafted with Claude Code (claude opus 5 (high)), which wrote the script, ran the freeze, and reviewed the result over three adversarial review rounds.

The `.. ipython::` blocks in the whatsnew files are re-executed on every doc
build against whatever pandas is current, so they illustrate behavior other
than the release they describe, and they periodically break the
warnings-as-errors doc build when a deprecation they demonstrate is enforced.

Add scripts/freeze_whatsnew.py, which rewrites those directives as static
`.. code-block:: pycon` sessions, and apply it to all 45 files that had
executable blocks. For 0.17.0 and newer the script reads the file back out of
git at that release's tag and runs it against that release's pandas, fetched
with `pixi exec`, so the frozen output is what the release produced rather than
what pandas does today. conda-forge has no pandas below 0.17.1, so the 16
pre-0.17 files are rendered against the current environment instead.

799 executable blocks and 49 :okexcept:/:okwarning: markers go to zero.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jbrockmendel
jbrockmendel marked this pull request as ready for review September 1, 2026 01:52

@mroeschke mroeschke left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I think it would be helpful to have a pre-commit check for the doc/source/whatsnew directory to ensure .. code-block: pycon is the only allowed code-block directive in these files
  2. IMO I don't think we'll need to commit freeze_whatsnew.py and the test it's essentially a one time conversion, and we should ensure all whatsnew docs (even the current) are written in a "frozen" state"

jbrockmendel and others added 2 commits September 1, 2026 11:44
Per review: the conversion tool is not worth committing. It shells out to
pixi for pandas 0.17.1-2.x under CPython 3.5/3.6, so it would rot in
scripts/ and run 30 tests in CI for something nobody will invoke again.
It stays reachable in this PR's history at 82d9c06.

Replace it with a pygrep hook rejecting any `.. ipython` directive under
doc/source/whatsnew/, which keeps every page frozen going forward,
including the in-development v3.1.0.

The hook also matches the missing-colons spelling, which caught the
pre-existing v0.23.0 `.. ipython` typo that docutils was swallowing as a
comment; its sort=False example is now frozen like its two siblings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVo28HBquFdK4ZmkjX3QZs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DOC: freeze old whatsnew notes?

2 participants