fix comments being silently dropped when a collection is rendered inline#67
Conversation
The dumper promises (doc/comments.rst, releases.rst) that comments are
re-emitted when the same keymap is supplied. The multiline path already
guards this via _comments_force_multiline, forcing a value onto its own line
so its comments are not lost. The inline path (render_value's {...}/[...]
branches) had no such guard, so any comment on an inlined collection's value
slots or on its descendants was dropped with no warning.
Add _inline_would_drop_comments, mirroring _comments_force_multiline, and
raise NotSuitableForInline (the existing inline->multiline fallback) when
inlining would drop a comment.
|
Oh, your code introduces an N-squared loop into the dumper. I'll have to spend some time thinking about this one. |
_inline_would_drop_comments rescanned all of map_keys for every collection considered for inlining, which is O(N^2) over the document. Build the set of unsafe-to-inline keys in a single pass over map_keys and cache it on the dumper, so each check is an O(1) membership test. Behaviour is unchanged: the comment suite (139) and the official-corpus round-trip tests (1837) pass, and a deep-comment dump now scales linearly with document size instead of quadratically.
|
Good catch — fixed in 040b299. I replaced the per-collection rescan with a single pass over |
|
Your approach is not quite right. There is no need to examine the hierarchy in advance to determine whether there are comments that would interfere with the inlining. If you move your _inline_would_drop_comments test from render_value to the render_inline_... functions, the validity would be checked at each level of the hierarchy, and a comment at any level would break out of the inlining. Also, I'm not sure your understanding of the providers is correct. Provider from the parent contribute to potential comments on its children. I am going to accept your pull request, because I think that by identifying this flaw in the program, you have made an important contribution, and so your name should be on the project, but at this point it looks unlikely that I will be using any of your code. Hold off doing anything more for now. I already have a first cut using the approach described above. And I want to build a comprehensive example as a test case so we can get this right. I'll get back to you once I check those in. And thank you. |
The problem
doc/comments.rstand the current release notes both state that the dumper, given the same keymap, re-emits each comment in its place. That holds for the multiline form but not for the inline form: whenwidthandinline_level/inline_countmake a collection eligible for inline rendering, comments on that collection's value slots and on all of its descendants are dropped silently.The fix
The multiline path already protects against this:
_comments_force_multilineforces a value onto its own line "so those comments don't get silently dropped". The inline branches ofrender_valuejust never applied an equivalent check.This adds
_inline_would_drop_comments(keys), modelled on_comments_force_multiline, and raises the existingNotSuitableForInline(the normal inline to multiline fallback) when inlining would drop a comment. It checks value-side comments/providers at the collection itself (key-side comments there are still emitted by the parent, so they don't force multiline) and any comment or provider on a strict descendant. There's a no-op fast path whenmap_keysisn't a keymap, so the common no-comments case is unaffected.So an inlined collection that carries comments now falls back to the multiline form and keeps them; collections with no comments still inline exactly as before.
Tests
Two tests added to
tests/test_comments.py(list-item key comment, dict value-trailing comment); each fails before the change (comment dropped) and passes after. Full suite: 2180 passed, 680 skipped;ruff checkclean.I left
doc/releases.rstuntouched since this completes the in-development comment feature rather than changing released behaviour. Happy to add a note there, or to adjust the approach if you had a different intent for how inline rendering and comments should interact.I used an AI assistant to help investigate this under my direction, and I have reviewed and verified the change myself.