Skip to content

Commit d0bcbe0

Browse files
committed
feat: add diff command for tracking compatibility progress over time
- Add ftready diff subcommand to compare two JSON reports - Refactor CLI from single command to click group with default 'check' subcommand - Show improved/regressed/added/removed packages with delta counts - Support text and JSON output for diffs (--format text|json) - Export DiffSummary, PackageDiff, diff_reports, format_diff from public API - Update README: PEP 703 link, pure-Python explanation, --cache-ttl/--cache-file docs, tracking progress section, Python API examples, limitations section, uv install option - Add 23 new tests for diff module and CLI subcommand (129 total, 93% coverage)
1 parent 2571afc commit d0bcbe0

5 files changed

Lines changed: 724 additions & 18 deletions

File tree

README.md

Lines changed: 99 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -10,21 +10,23 @@
1010
<a href="https://github.com/dxvidparham/ftready/actions/workflows/ci.yml"><img src="https://github.com/dxvidparham/ftready/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
1111
<a href="https://pypi.org/project/ftready/"><img src="https://img.shields.io/pypi/v/ftready" alt="PyPI version"></a>
1212
<a href="https://pypi.org/project/ftready/"><img src="https://img.shields.io/pypi/pyversions/ftready" alt="Python versions"></a>
13-
<a href="https://github.com/dxvidparham/ftready"><img src="https://img.shields.io/badge/coverage-92%25-brightgreen" alt="Coverage"></a>
13+
<a href="https://github.com/dxvidparham/ftready"><img src="https://img.shields.io/badge/coverage-93%25-brightgreen" alt="Coverage"></a>
1414
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License: MIT"></a>
1515
</p>
1616

1717
<p align="center">
1818
<a href="#-quickstart">Quickstart</a> ·
1919
<a href="#-features">Features</a> ·
2020
<a href="#-usage">Usage</a> ·
21+
<a href="#-tracking-progress">Tracking Progress</a> ·
2122
<a href="#-ci-integration">CI Integration</a> ·
23+
<a href="#-python-api">Python API</a> ·
2224
<a href="https://pypi.org/project/ftready/">PyPI</a>
2325
</p>
2426

2527
---
2628

27-
Python 3.13 shipped a **free-threaded build** (`-t` suffix) that disables the GIL — unlocking true multi-core parallelism. But your project is only as ready as its **least-compatible dependency**.
29+
[PEP 703](https://peps.python.org/pep-0703/) introduced a **free-threaded build** of CPython (3.13t / 3.14t) that disables the GIL — unlocking true multi-core parallelism. But your project is only as ready as its **least-compatible dependency**. See the [CPython free-threading docs](https://docs.python.org/3.14/whatsnew/3.13.html#free-threaded-cpython) for background.
2830

2931
`ftready` scans your dependency tree, queries PyPI for `cp313t`/`cp314t` wheels, cross-references [ft-checker.com](https://ft-checker.com) test results, and tells you exactly where you stand:
3032

@@ -47,7 +49,7 @@ Python 3.13 shipped a **free-threaded build** (`-t` suffix) that disables the GI
4749
## ⚡ Quickstart
4850

4951
```bash
50-
pip install ftready
52+
pip install ftready # or: uv tool install ftready
5153
ftready
5254
```
5355

@@ -60,6 +62,7 @@ That's it. Reads your `pyproject.toml` and prints a compatibility report.
6062
- **Pure-Python detection** — flags packages with no C extensions as likely compatible
6163
- **Every input format**`pyproject.toml` (PEP 621 + Poetry), `requirements.txt`, `uv.lock`, `poetry.lock`, `pdm.lock`
6264
- **Full dependency tree**`--all-deps` scans transitive deps via lock files with pinned-version accuracy
65+
- **Progress tracking**`ftready diff` compares two JSON reports to show what changed over time
6366
- **Multiple outputs** — rich tables, plain text, JSON, or CSV
6467
- **CI-ready** — configurable exit codes (`--fail-on`) to gate or report without blocking
6568
- **Offline-safe**`--no-ftchecker` and `--no-pypi` flags for controlled environments
@@ -85,30 +88,62 @@ ftready --plain # plain-text table
8588
ftready --format json # JSON (for scripting)
8689
ftready --format csv # CSV
8790
ftready --output report.txt # write to file
91+
ftready -v # verbose progress to stderr
8892
```
8993

9094
### Data source control
9195

9296
```bash
93-
ftready --no-ftchecker # PyPI only (skip ft-checker.com)
94-
ftready --no-pypi # ft-checker only (skip PyPI)
95-
ftready --no-cache # force fresh ft-checker scrape
97+
ftready --no-ftchecker # PyPI only (skip ft-checker.com)
98+
ftready --no-pypi # ft-checker only (skip PyPI)
99+
ftready --no-cache # force fresh ft-checker scrape
100+
ftready --cache-ttl 48 # cache ft-checker data for 48 hours
101+
ftready --cache-file /tmp/ft_cache.json # custom cache location
96102
```
97103

98104
### Exit codes
99105

100-
| Code | Meaning |
101-
| ---- | ------- |
106+
| Code | Meaning |
107+
| ---- | ------------------------------------------------- |
102108
| `0` | No blocking issues (configurable via `--fail-on`) |
103-
| `1` | At least one dependency has a **Failed** status |
104-
| `2` | Configuration error (missing input file, etc.) |
109+
| `1` | At least one dependency has a **Failed** status |
110+
| `2` | Configuration error (missing input file, etc.) |
105111

106112
```bash
107113
ftready --fail-on=never # always exit 0 (report only)
108114
ftready --fail-on=unknown # exit 1 on Failed OR Unknown
109115
ftready --fail-on=failed # exit 1 on Failed only (default)
110116
```
111117

118+
## 📈 Tracking Progress
119+
120+
Save JSON reports over time and use `ftready diff` to see what changed:
121+
122+
```bash
123+
# Save a baseline
124+
ftready --format json --output baseline.json
125+
126+
# ... weeks later, check again
127+
ftready --format json --output current.json
128+
129+
# Compare the two
130+
ftready diff baseline.json current.json
131+
```
132+
133+
```
134+
ftready diff: 2025-01-15T12:00:00+00:00 → 2025-03-01T12:00:00+00:00
135+
136+
3.13t ready: 5 → 8 (+3)
137+
3.14t ready: 2 → 5 (+3)
138+
139+
✅ Improved (3):
140+
scipy: 3.13t: Not tested → Success
141+
pillow: 3.13t: Not tested → Success, 3.14t: Not tested → Success
142+
matplotlib: 3.14t: Not tested → Success
143+
```
144+
145+
The diff also supports `--format json` for machine-readable output.
146+
112147
## 🔄 How It Works
113148

114149
```
@@ -126,6 +161,8 @@ uv.lock ────────┘ │ (parallel) (optional)
126161
4. **Enrich** — optionally fetches [ft-checker.com](https://ft-checker.com) test results (cached 24h)
127162
5. **Report** — renders results as a styled table, JSON, or CSV with configurable exit codes
128163

164+
> **About "Pure Python":** A package is flagged as 🐍 when *all* its published wheels use the `py3-none-any` tag — meaning no compiled C extensions. These packages typically work on free-threaded Python without changes. However, packages using `ctypes`, `cffi`, or calling native code at runtime may still have issues despite being "pure." Treat this as a strong signal, not a guarantee.
165+
129166
## 🏗 CI Integration
130167

131168
**Report without blocking:**
@@ -160,16 +197,63 @@ uv.lock ────────┘ │ (parallel) (optional)
160197
path: ft-report.json
161198
```
162199
200+
**Track progress across CI runs** by downloading a previous artifact and diffing:
201+
202+
```yaml
203+
- name: Compare with baseline
204+
run: |
205+
pip install ftready
206+
ftready --format json --output current.json --fail-on=never
207+
ftready diff baseline.json current.json || true
208+
```
209+
210+
## 🐍 Python API
211+
212+
`ftready` exports a public API for programmatic use:
213+
214+
```python
215+
from ftready import (
216+
load_dependencies,
217+
fetch_ftchecker_db,
218+
build_results,
219+
generate_report,
220+
diff_reports,
221+
format_diff,
222+
)
223+
from pathlib import Path
224+
225+
# Load deps from pyproject.toml
226+
deps = load_dependencies(Path("pyproject.toml"))
227+
228+
# Query data sources
229+
ft_db = fetch_ftchecker_db(Path(".ft_cache.json"))
230+
results = build_results(deps, ft_db)
231+
232+
# Render as JSON
233+
report = generate_report(results, include_dev=False, output_format="json")
234+
235+
# Compare two JSON reports
236+
summary = diff_reports("old_report.json", "new_report.json")
237+
print(format_diff(summary))
238+
```
239+
163240
## 📊 Data Sources
164241

165-
| Source | Role | Coverage |
166-
| ------ | ---- | -------- |
167-
| PyPI JSON API | **Primary** — `cp313t`/`cp314t` wheel tag detection | Every package on PyPI |
168-
| PyPI version endpoint | **Primary** — pinned-version checking from lock files | Every package on PyPI |
169-
| [ft-checker.com](https://ft-checker.com) | **Enrichment** — actual test results | ~1000 top packages |
242+
| Source | Role | Coverage |
243+
| ---------------------------------------- | ----------------------------------------------------- | --------------------- |
244+
| PyPI JSON API | **Primary** — `cp313t`/`cp314t` wheel tag detection | Every package on PyPI |
245+
| PyPI version endpoint | **Primary** — pinned-version checking from lock files | Every package on PyPI |
246+
| [ft-checker.com](https://ft-checker.com) | **Enrichment** — actual test results | ~1000 top packages |
170247

171248
When both sources report on a package, ft-checker.com takes priority — a package may ship free-threaded wheels but still fail tests.
172249

250+
## ⚠️ Limitations
251+
252+
- **Latest release only** (without lock files): When no lock file is used, ftready checks the *latest* PyPI release. If you're pinned to an older version, use `--all-deps` with a lock file for exact version checks.
253+
- **Wheel tags ≠ runtime compatibility**: A package shipping `cp313t` wheels doesn't guarantee it works correctly under free-threaded Python — it only means the maintainer built wheels for that target. ft-checker.com test results provide stronger evidence.
254+
- **ft-checker.com coverage**: The enrichment source covers ~1000 popular packages. Niche packages may only have PyPI wheel tag data.
255+
- **No transitive blocker tracing**: The tool reports status per-package but doesn't show dependency chains (e.g., "X depends on Y which depends on Z, and Z is the blocker").
256+
173257
## 📄 License
174258

175259
[MIT](LICENSE)

src/ftready/__init__.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,17 +2,23 @@
22

33
from ftready.checker import build_results
44
from ftready.constants import Status
5+
from ftready.diff import DiffSummary, PackageDiff, diff_reports, format_diff, format_diff_json
56
from ftready.models import PackageResult
67
from ftready.parser import load_dependencies, load_lockfile_dependencies, load_requirements, load_uv_lock_dependencies
78
from ftready.report import generate_report
89
from ftready.scraper import check_pypi_batch, fetch_ftchecker_db
910

1011
__all__ = [
12+
"DiffSummary",
13+
"PackageDiff",
1114
"PackageResult",
1215
"Status",
1316
"build_results",
1417
"check_pypi_batch",
18+
"diff_reports",
1519
"fetch_ftchecker_db",
20+
"format_diff",
21+
"format_diff_json",
1622
"generate_report",
1723
"load_dependencies",
1824
"load_lockfile_dependencies",

src/ftready/cli.py

Lines changed: 50 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111

1212
from ftready.checker import build_results
1313
from ftready.constants import _DEFAULT_CACHE_TTL_HOURS, STATUS_FAILED, STATUS_UNKNOWN
14+
from ftready.diff import diff_reports, format_diff, format_diff_json
1415
from ftready.parser import load_dependencies, load_lockfile_dependencies, load_requirements, load_uv_lock_dependencies
1516
from ftready.report import generate_report
1617
from ftready.scraper import fetch_ftchecker_db
@@ -102,7 +103,24 @@ def _resolve_deps(
102103
return deps, direct_names
103104

104105

105-
@click.command(help="Check project dependencies for free-threaded Python (3.13t/3.14t) compatibility.")
106+
class _DefaultGroup(click.RichGroup):
107+
"""A click Group that falls through to a default subcommand when none is given."""
108+
109+
default_cmd_name: str = "check"
110+
111+
def parse_args(self, ctx: click.Context, args: list[str]) -> list[str]:
112+
"""Prepend the default subcommand when args don't start with a known command."""
113+
if not args or args[0] not in self.commands:
114+
args = [self.default_cmd_name, *args]
115+
return super().parse_args(ctx, args)
116+
117+
118+
@click.group(cls=_DefaultGroup, help="Check project dependencies for free-threaded Python (3.13t/3.14t) compatibility.")
119+
def main() -> None:
120+
"""Entry point group — delegates to ``check`` by default."""
121+
122+
123+
@main.command(help="Check project dependencies for free-threaded Python compatibility.")
106124
@click.option(
107125
"--pyproject",
108126
type=click.Path(path_type=Path),
@@ -166,7 +184,7 @@ def _resolve_deps(
166184
show_default=True,
167185
help="When to exit with code 1.",
168186
)
169-
def main(
187+
def check(
170188
pyproject: Path,
171189
output: Path | None,
172190
cache_file: Path,
@@ -183,7 +201,7 @@ def main(
183201
requirements: Path | None,
184202
fail_on: str,
185203
) -> None:
186-
"""Entry point for the free-threaded compatibility checker."""
204+
"""Run the free-threaded compatibility check."""
187205
if verbose:
188206
logging.basicConfig(level=logging.INFO, format="%(message)s", stream=sys.stderr, force=True)
189207

@@ -237,3 +255,32 @@ def main(
237255
)
238256
if any_direct_bad:
239257
sys.exit(1)
258+
259+
260+
@main.command(name="diff", help="Compare two JSON reports to track compatibility changes over time.")
261+
@click.argument("old_report", type=click.Path(exists=True, path_type=Path))
262+
@click.argument("new_report", type=click.Path(exists=True, path_type=Path))
263+
@click.option(
264+
"--format",
265+
"output_format",
266+
type=click.Choice(["text", "json"]),
267+
default="text",
268+
show_default=True,
269+
help="Output format for the diff.",
270+
)
271+
@click.option("--output", type=click.Path(path_type=Path), default=None, help="Write the diff to this file.")
272+
def diff_cmd(
273+
old_report: Path,
274+
new_report: Path,
275+
output_format: str,
276+
output: Path | None,
277+
) -> None:
278+
"""Compare two ftready JSON reports and show what changed."""
279+
summary = diff_reports(str(old_report), str(new_report))
280+
result = format_diff_json(summary) if output_format == "json" else format_diff(summary)
281+
282+
if output:
283+
output.write_text(result, encoding="utf-8")
284+
click.echo(f"Diff written to {output}", err=True)
285+
else:
286+
click.echo(result)

0 commit comments

Comments
 (0)