Run the automation tooling tests in CI (#4939)

Three test suites live in the repository, pass, and are wired to no workflow,
so a regression in any of them reaches main silently:

  .agents/skills/ci-status/scripts/tests/        8 tests
  .agents/skills/update-skia/scripts/*_test.py   4 modules
  .github/scripts/tests/skia-sync-detect.test.sh

The ci-status registry suite is the sharpest example. It exists to catch a
dashboard that reports a workflow which is not there, but nothing ran it — so
the guard itself was unguarded.

Adds a workflow covering all three, scoped tightly to the paths that feed them
plus any .github/workflows/*.yml edit, which is what makes the registry suite
re-check its tracked list whenever a workflow is added, renamed or deleted.

The ci-status suite is invoked directly rather than behind an existence check.
On the runner's Python, unittest discover already fails loudly in both
degenerate cases — exit 1 if the directory is gone, exit 5 on "NO TESTS RAN"
if it is empty — so a wrapper could only convert those into a silent pass. The
update-skia loop does need an explicit guard, because a glob that matches
nothing would otherwise iterate zero times and succeed.

Also registers the new workflow with the ci-status dashboard it runs, keeping
coverage at 27 of 29.

Co-authored-by: v <v@v.v>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Author: 3 people

.agents/skills/ci-status/SKILL.md

@@ -86,6 +86,7 @@ standard Darc/Maestro stages.
 | Release - Milestones | mono/SkiaSharp | Workflow dispatch | Milestone reconciliation stops |
 | Update GitHub Release summaries | mono/SkiaSharp | Workflow dispatch | Release bodies go stale |
 | Release - Tooling Tests | mono/SkiaSharp | Push/PR to release tooling | Release scripts regress unnoticed |
+| Automation - Tooling Tests | mono/SkiaSharp | Push/PR to automation tooling | Automation scripts regress unnoticed |
 | PR - Backport | mono/SkiaSharp | PR label/comment | Cherry-picks to release branches fail |
 | PR - Rebase | mono/SkiaSharp | PR comment | PR rebase automation broken |
 | PR - Artifacts Comment | mono/SkiaSharp | Workflow run events | Build links not posted to PRs |
@@ -247,7 +248,7 @@ For each tracked GitHub Actions workflow:
     (broken = user-facing impact or release process blocked)
   - **Medium**: Sync - Docs Submodule, Sync - Skia Submodule, Fixer - Memory Leak, Fixer - Performance,
     Sync - Issue Triage, Sync - Issue Template Versions, Tests - Binding Generation Determinism,
-    Release - Milestones, Update GitHub Release summaries,
+    Automation - Tooling Tests, Release - Milestones, Update GitHub Release summaries,
     PR - Backport, Pages - Go Live! (broken = automation degraded, manual workaround exists)
   - **Low**: Pages - PR Staging - Cleanup, Pages - PR Staging - Sweep Stale, PR - Rebase, PR - Artifacts Comment,
     Merge Message, Track - Artifact Sizes, Track - Benchmarks,

.agents/skills/ci-status/scripts/ci-status.py

@@ -89,6 +89,7 @@
     {"repo": "mono/SkiaSharp", "workflow": "release-milestones.yml", "name": "Release - Milestones", "scope": "global", "trigger": "dispatch"},
     {"repo": "mono/SkiaSharp", "workflow": "update-github-release-summaries.yml", "name": "Update GitHub Release summaries", "scope": "global", "trigger": "dispatch"},
     {"repo": "mono/SkiaSharp", "workflow": "release-tooling-tests.yml", "name": "Release - Tooling Tests", "scope": "branch", "trigger": "push"},
+    {"repo": "mono/SkiaSharp", "workflow": "automation-tooling-tests.yml", "name": "Automation - Tooling Tests", "scope": "branch", "trigger": "push"},
     # mono/SkiaSharp — PR Utilities (global: triggered by PR events, not branch-specific)
     {"repo": "mono/SkiaSharp", "workflow": "backport.yml", "name": "PR - Backport", "scope": "global", "trigger": "event"},
     {"repo": "mono/SkiaSharp", "workflow": "rebase.yml", "name": "PR - Rebase", "scope": "global", "trigger": "event"},

.github/workflows/automation-tooling-tests.yml

@@ -0,0 +1,80 @@
+name: Automation - Tooling Tests
+
+# Runs the repository's non-release automation test suites. These all existed and passed
+# locally but were wired to no workflow, so a regression in them could reach `main`
+# silently:
+#
+#   * .agents/skills/ci-status/scripts/tests/       — the CI dashboard's workflow registry
+#   * .agents/skills/update-skia/scripts/*_test.py  — Skia milestone update tooling
+#   * .github/scripts/tests/skia-sync-detect.test.sh — the auto-skia-sync work detector
+#
+# Release publishing tooling is covered separately by release-tooling-tests.yml. Paths are
+# kept tight on purpose: this must not become a catch-all that runs on every change.
+
+on:
+  pull_request:
+    paths:
+      - ".agents/skills/ci-status/**"
+      - ".agents/skills/update-skia/**"
+      - ".github/scripts/**"
+      # Any workflow edit, so the registry test re-checks its tracked list. '*.yml' already
+      # covers the generated '*.lock.yml' files.
+      - ".github/workflows/*.yml"
+  push:
+    branches: [main]
+    paths:
+      - ".agents/skills/ci-status/**"
+      - ".agents/skills/update-skia/**"
+      - ".github/scripts/**"
+      - ".github/workflows/*.yml"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: automation-tooling-tests-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - name: Checkout
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+
+      - name: Run ci-status workflow registry tests
+        # Guards the CI dashboard against tracking workflows that no longer exist — a stale
+        # entry does not error, it renders a GREEN row for a workflow that is not there.
+        #
+        # Called directly rather than behind an existence check: on the runner's Python
+        # (3.12+) `unittest discover` fails loudly in both degenerate cases — exit 1
+        # (ImportError) if the directory is gone, exit 5 ("NO TESTS RAN") if it is empty.
+        # A guard here would only be able to turn those into a silent pass.
+        run: |
+          set -euo pipefail
+          python3 -c "import yaml" 2>/dev/null \
+            || python3 -m pip install --quiet --break-system-packages pyyaml
+          python3 -m unittest discover -s .agents/skills/ci-status/scripts/tests -p "test_*.py" -v
+
+      - name: Run update-skia tooling tests
+        # These import their module under test by name, so they must run from their own
+        # directory rather than through a repository-root discovery run.
+        working-directory: .agents/skills/update-skia/scripts
+        run: |
+          set -euo pipefail
+          shopt -s nullglob
+          tests=(*_test.py)
+          if [ ${#tests[@]} -eq 0 ]; then
+            echo "::error::No *_test.py found — the update-skia tests moved or were renamed."
+            exit 1
+          fi
+          for test in "${tests[@]}"; do
+            echo "::group::$test"
+            python3 -m unittest "${test%.py}"
+            echo "::endgroup::"
+          done
+
+      - name: Run skia-sync work detector tests
+        run: bash .github/scripts/tests/skia-sync-detect.test.sh