Skip to content

feat(kg-build): surface URL handler and mapping failures with typed result and sync report - #19

Merged
rpanfili merged 18 commits into
mainfrom
fail_run
May 29, 2026
Merged

feat(kg-build): surface URL handler and mapping failures with typed result and sync report#19
rpanfili merged 18 commits into
mainfrom
fail_run

Conversation

@rpanfili

@rpanfili rpanfili commented Apr 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Problem

URL handler failures (HTTP 403s, YARRRML parse errors) were silently swallowed —
the cloud workflow exited 0 with no record of what failed.

What changed

Failure capture

  • Introduced FailedUrl dataclass (timestamp, url, handler name, message) and
    KgImportResult (url count, failures, ok property, elapsed seconds) as the
    typed return value of KgImportWorkflow.run()
  • DefaultUrlHandler now accumulates failures per handler instead of logging and
    discarding them
  • rml_mapping.apply_mapping now re-raises on processing errors (YARRRML parse,
    missing dataset URI) instead of returning MappingResult(graph=None), so
    mapping failures propagate through the protocol into KgImportResult.failures
    alongside HTTP failures
  • run_cloud_workflow and main.run_kg_import_workflow both raise
    SystemExit("Total URLs: X, Successes: Y, Failures: Z") on failure

Reporting

  • New report_util module: render_as_markdown, render_as_csv, write_report
  • When CloudWorkflowConfig.output_dir is set, writes graph_sync_report.md
    (always) and graph_sync_failures.csv (on failure only)
  • Markdown report: summary table with total URLs, successes, failures, success
    rate, and execution time; top-10 error groups with count, stable key, clickable
    example URL, and full error message; overflow note when distinct error types
    exceed the limit
  • CSV: timestamp, url, error per failed URL

@rpanfili
rpanfili force-pushed the fail_run branch 5 times, most recently from 1bb4dd8 to 51b9b2b Compare April 2, 2026 15:43
@rpanfili rpanfili changed the title Fail graph import run if any URL handler fails feat(kg-build): surface URL handler and mapping failures with typed result and sync report May 29, 2026
@rpanfili
rpanfili marked this pull request as ready for review May 29, 2026 14:56
@rpanfili
rpanfili merged commit b57b3f7 into main May 29, 2026
12 checks passed
@rpanfili
rpanfili deleted the fail_run branch May 29, 2026 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant