fix: Do not choose top node that will be emptied by post_cleanup

[Copilot]

calculate_best_node always selected parent_nodes[0] (highest gravity score) without checking whether that node has any actual text content. Nodes can score highly via grandparent propagation while being structurally empty (images-only, empty wrappers, etc.), resulting in a top node that post-cleanup strips entirely—producing an empty article.

Changes

• newspaper/extractors/articlebody_extractor.py: calculate_best_node now iterates through sorted candidates and picks the first with non-empty text (parsers.get_text(candidate)), skipping nodes that would be emptied by cleanup. Returns None if no candidate has text.

# Before
top_node = parent_nodes[0]

# After
for candidate in parent_nodes:
    if parsers.get_text(candidate):
        top_node = candidate
        break

• tests/unit/test_parsing.py: Added TestArticleBodyExtractor with two tests covering the skip-empty behavior and the all-empty-candidates → None fallback.

How did you test it?

Unit tests via pytest tests/unit/test_parsing.py. The new tests mock compute_gravity_scores to inject an empty high-score node alongside a text-containing lower-score node, verifying the correct node is selected.

Notes for the reviewer

parsers.get_text() already calls inner_trim (strips all whitespace), so the truthiness check is sufficient—no additional regex stripping needed.

Checklist

• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
• I documented my code
• I ran pre-commit hooks and fixed any issue

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• example.com
  • Triggering command: /home/REDACTED/.local/bin/pytest pytest tests/unit/ --ignore=tests/unit/test_google_news.py --ignore=tests/unit/test_nlp.py -x -q (dns block)
  • Triggering command: /home/REDACTED/.local/bin/pytest pytest tests/unit/ --ignore=tests/unit/test_google_news.py --ignore=tests/unit/test_nlp.py -q (dns block)
  • Triggering command: /home/REDACTED/.local/bin/pytest pytest tests/unit/test_article.py tests/unit/test_cleaners.py tests/unit/test_parsing.py -v (dns block)
• media.cnn.com
  • Triggering command: /home/REDACTED/.local/bin/pytest pytest tests/unit/ --ignore=tests/unit/test_google_news.py --ignore=tests/unit/test_nlp.py -q (dns block)
• publicsuffix.org
  • Triggering command: /home/REDACTED/.local/bin/pytest pytest tests/unit/ --ignore=tests/unit/test_google_news.py --ignore=tests/unit/test_nlp.py -q (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)