Skip to content

Phase 1: CI/CD pipeline, dead link cleanup, and URL normalization - #546

Merged
s0lray merged 1 commit into
masterfrom
phase-1/ci-and-dead-links
Mar 22, 2026
Merged

Phase 1: CI/CD pipeline, dead link cleanup, and URL normalization#546
s0lray merged 1 commit into
masterfrom
phase-1/ci-and-dead-links

Conversation

@s0lray

@s0lray s0lray commented Mar 22, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR delivers Phase 1 infrastructure improvements to the OSINT Framework:

Changes

  • CI/CD Pipeline (.github/workflows/ci.yml): Added GitHub Actions with three jobs:

    • JSON lint: validates arf.json is well-formed on every PR and push
    • Smoke test: verifies index.html local asset references exist
    • Link check: weekly scheduled + master push Lychee scan of all URLs
  • Dead link removal: Removed 113 confirmed dead links from arf.json using automated tooling (tools/link_checker.py, tools/remove_dead_links.py)

  • URL normalization: Upgraded ~215 URLs from http:// to https:// where redirects were detected

  • Tooling added (tools/): Link checker and dead link remover scripts for ongoing maintenance

  • .gitignore: Added agents/ directory to ignore internal agent config files

Results

Metric Before After
Total URLs 1,113 1,000
Dead links 283 169
HTTP → HTTPS upgrades ~215

Test Plan

  • CI JSON lint passes on this PR
  • Smoke test passes
  • Manual spot-check removed URLs are indeed dead
  • No live links were accidentally removed

Rebased cleanly onto current master (resolving conflicts from
community PRs #240-#471 merged while this branch was in flight).

Changes:
- Add GitHub Actions CI workflow (JSON lint, link check, smoke tests)
- Remove 176 confirmed dead links from arf.json
- Normalize 393 URLs from http:// to https://
- Fix invalid JSON (missing commas/braces from old format)
- Remove Bower dependency; lock D3 to npm
- Add link audit tooling (tools/)
- Cherry-pick alive tools from community PR #391:
  FaceCheckID, Cisco Talos, VerifyEmail, YouTube Geofind, Whitepages,
  Hiya, Berla Vehicle Lookup, OpenAIP, Valkyrie File Analysis
- Fix OpenInfrastructureMap URL (missing https://)

Community PR #359 (Risk Toolbox / fraudfightersalliance.com):
DNS does not resolve — all 27 proposed additions are dead, not merged.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
@s0lray
s0lray force-pushed the phase-1/ci-and-dead-links branch from a3bb071 to 8c2de4d Compare March 22, 2026 03:00
@s0lray
s0lray merged commit e21740a into master Mar 22, 2026
3 checks passed
s0lray added a commit that referenced this pull request Mar 22, 2026
…nk cleanup)

All 9 entries cherry-picked in 059a8b4 are already present in the remote
version (they were independently merged via PR #546). Resolving by accepting
theirs for arf.json, keeping staged CSS/HTML changes from local branch.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
@s0lray
s0lray deleted the phase-1/ci-and-dead-links branch March 24, 2026 00:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant