Skip to content

Commit e0084ea

Browse files
authored
docs(readme): sync status banner and roadmap to v0.2.0 (#27)
* docs(readme): update status banner and roadmap for v0.2.0 The status line and Status-and-roadmap section still referenced 0.1.0 after the v0.2.0 release shipped. Updates: - Status banner now reads "0.2.0 - Phase 1 first wave shipped" and names the five additions (inspect, history, paraphrase, case study, CD pipeline). - Roadmap section reframes 0.2.0 as the current release with all five items checked off; 0.1.0 stays as the historical Phase-0 MVP entry. - "Coming next" section names the two locked tracks: diff sharpening (Phase D) and the post-Phase-D artifact infrastructure track (verify -> export --bundle -> embedded invocation). Lean by design; not pre-planning beyond Phase D. * docs: sync release-state metadata for v0.2.0 Pre-merge audit caught several stale 0.1.0 references that should be 0.2.0 now that the release has shipped. All release-state metadata (machine-readable + reader-facing): - CITATION.cff: version 0.1.0 -> 0.2.0; date-released 2026-05-21 -> 2026-05-22. - CHANGELOG.md: add [0.2.0] reference link at the bottom (matches the existing [0.1.0] link pattern). - docs/EVIDENCE.md: "Today (0.1.0)" -> "Today (0.2.0)" in the artifact-identity section; also names the post-Phase-D artifact track (verify -> export --bundle) since it's the concrete answer to what's "deferred" in that paragraph. - docs/case-studies/01-invisible-character-substitution.md: pip install command updated from 0.1.0 to 0.2.0 so readers install the version that contains the case study itself. - docs/RELEASE.md: example tag commands use v0.2.0 instead of v0.1.0; GitHub release section uses generic <version>/<thematic-name> placeholders; "next dev marker" section advances the example from 0.2.0.dev0 to 0.3.0.dev0 and adds tests/unit/test_version.py to the bump checklist (caught by the actual 0.2.0 release). Left alone (historically accurate): - docs/case-studies/data/README.md "FalsifyAI version: 0.1.0" - provenance for when those Phase-0 campaign sessions were generated. - docs/ARCHITECTURE.md "between 0.1.0 and 1.0" - resolver discipline statement, accurate from 0.1.0 baseline. - CHANGELOG context lines mentioning "unchanged from 0.1.0" - accurate historical context within the 0.2.0 entry. These commits will land in PR #27 (README sync to v0.2.0) since both are pre-merge release-state metadata cleanup. * docs(release): add release-state metadata audit checklist Expands item 7 of the pre-release checklist into an explicit audit of drift-prone version references, capturing the lesson from the v0.2.0 release where five files needed metadata syncs after the tag landed (CITATION.cff, README status banner, EVIDENCE.md "Today (X.Y.Z)" assertion, case study pip-install commands, CHANGELOG link). Names the specific files that contain release-state metadata and must advance on every release, AND distinguishes them from files that contain historically-accurate version references that must NOT change (provenance, "unchanged from 0.1.0" context, baseline discipline statements). Deliberately a human-judgment checklist, not a CI gate: a blanket grep for version strings would produce both signals (release-state and historical context) with no way to distinguish, and an automated rule would erode trust faster than it saved time. The next release pays a 30-second deliberate check instead of a 30-minute audit.
1 parent d318d89 commit e0084ea

6 files changed

Lines changed: 73 additions & 27 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -287,4 +287,5 @@ All four are verified in CI via `tests/integration/test_examples.py`.
287287
- **`--latest-baseline` / `--latest-candidate`** flags on `diff` are not
288288
shipped; users pass explicit session ids. Phase 1 candidate.
289289

290+
[0.2.0]: https://github.com/ericckzhou/falsifyai/releases/tag/v0.2.0
290291
[0.1.0]: https://github.com/ericckzhou/falsifyai/releases/tag/v0.1.0

CITATION.cff

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,5 +26,5 @@ keywords:
2626
- reliability-testing
2727
- AI-safety
2828
license: Apache-2.0
29-
version: 0.1.0
30-
date-released: '2026-05-21'
29+
version: 0.2.0
30+
date-released: '2026-05-22'

README.md

Lines changed: 13 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ Most evaluation tools produce metrics. FalsifyAI produces **evidence** — durab
88
[![Python](https://img.shields.io/badge/python-3.13%2B-blue)](https://www.python.org)
99
[![License](https://img.shields.io/badge/license-Apache%202.0-green)](LICENSE)
1010

11-
**Status:** 0.1.0 — Phase 0 MVP. Stable enough to use; spec language and verdict semantics are locked for the 0.1.x line.
11+
**Status:** 0.2.0 — Phase 1 first wave shipped (`inspect`, `history`, `paraphrase`, canonical case study, automated PyPI publishing). Spec language and verdict semantics remain locked for the 0.x line.
1212

1313
```bash
1414
pip install falsifyai
@@ -432,15 +432,20 @@ Contributions follow the conventions in [`CONTRIBUTING.md`](CONTRIBUTING.md). Ar
432432

433433
## Status and roadmap
434434

435-
**0.1.0 (this release) — Phase 0 MVP.** Spec language, perturbation runtime, materializer, invariants, execution adapter, replay store, real verdict resolver (stratified bootstrap CI, CONSISTENTLY_WRONG, falsifiability scoring), and the three-command CLI (`run` + `replay` + `diff`).
435+
**0.2.0 (current release) — Phase 1 first wave.** Adds:
436436

437-
**Phase 1 in progress** (selected by evidence from real-world validation runs, not roadmap completeness):
437+
- ✅ **`falsifyai inspect <session_id>`** — per-case deep-dive over preserved evidence. Surfaces every perturbed input, output, and invariant judgment. `--case <case_id>` expands one case; `--full` disables truncation. Pure consumer surface — the artifact already contained the data.
438+
- ✅ **`paraphrase` perturbation family** — LLM-generated semantic-preserving rewrites with embedding-similarity validity gating. Tests semantic robustness as an orthogonal pressure axis to the character-level families. Configurable per-spec (`count`, `similarity_threshold`, `max_attempts`, optional `model` override).
439+
- ✅ **`falsifyai history <case_id>`** — temporal view of one case across saved sessions. Newest-first, one row per session, showing verdict + CI + worst family per row. Reads `case.verdict` from preserved artifacts; no aggregation, no trend inference, no reinterpretation.
440+
- ✅ **Canonical case study** — [Invisible character substitution](docs/case-studies/01-invisible-character-substitution.md): cross-model `contains`-contract brittleness as the thesis (`history`), Pair 3 model-migration regression as the vivid concrete proof (`diff` + `inspect`), over a [bundled replay artifact](docs/case-studies/data/case-study-replays.db) you can re-open and reproduce verbatim.
441+
- ✅ **Automated PyPI publishing via Trusted Publisher (OIDC)** — `.github/workflows/publish.yml` fires on any `v*` tag push: verifies version match, re-runs tests, builds, validates, publishes. No long-lived tokens in repo.
438442

439-
- ✅ **`falsifyai inspect <session_id>`** *(shipped)* — makes the replay artifact legible. Per-case deep-dive surfacing every perturbed input, every model output, every invariant judgment. `--case <case_id>` expands one case; `--full` disables truncation. Consumer-surface only; the artifact already contained the data.
440-
- ✅ **`paraphrase` perturbation family** *(shipped)* — LLM-generated semantic-preserving rewrites with embedding-similarity validity gating. Tests semantic robustness as an orthogonal pressure axis to the character-level families. Configurable per-spec (`count`, `similarity_threshold`, `max_attempts`, optional `model` override).
441-
- ✅ **`falsifyai history <case_id>`** *(shipped)* — temporal view of one case across saved sessions. Newest-first, one row per session, showing verdict + CI + worst family per row. Reads `case.verdict` from preserved artifacts; no aggregation, no trend inference, no reinterpretation.
442-
- ✅ **Canonical case study** *(shipped)* — [Invisible character substitution](docs/case-studies/01-invisible-character-substitution.md): the cross-model `contains`-contract brittleness pattern (`history`) plus the Pair 3 model migration regression (`diff` + `inspect`) over a [bundled replay artifact](docs/case-studies/data/case-study-replays.db) you can re-open and reproduce verbatim.
443-
- Hardened replay artifacts — cross-run lineage, immutable evidence semantics, and (eventually) signed bundles for cross-org transfer. These strengthen the existing artifact guarantees; the core differentiator remains the artifact's predictable semantics, not the wrapping.
443+
**0.1.0 — Phase 0 MVP.** Spec language, perturbation runtime, materializer, invariants, execution adapter, replay store, real verdict resolver (stratified bootstrap CI, CONSISTENTLY_WRONG, falsifiability scoring), and the three-command CLI (`run` + `replay` + `diff`).
444+
445+
**Coming next** — selected by evidence, not theoretical completeness:
446+
447+
- **`diff` sharpening** — `--strict`, `--show-trending`, exit code 6 for low-falsifiability gates. Tightens the binary regression criterion for users who want finer CI control without compromising resolver predictability.
448+
- **Artifact infrastructure track** — `falsifyai verify <session_id>` (integrity + provenance), `falsifyai export --bundle` (productize the case-study extraction pattern), and a persisted CLI-invocation field in `ReplayArtifact`. Locked sequence; reassess after a second case study or real user pressure.
444449

445450
Each addition is evaluated against: *does this preserve evidence density, resolver predictability, and the discipline that makes the artifact trustworthy?* See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md), [`docs/EVIDENCE.md`](docs/EVIDENCE.md), and [`CONTRIBUTING.md`](CONTRIBUTING.md) for the discipline.
446451

docs/EVIDENCE.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -324,10 +324,14 @@ first place. A reliability claim with predictable semantics is the
324324
contract; cryptographic provenance strengthens portability of that
325325
contract.
326326

327-
Today (0.1.0), the artifact has strong *deterministic* identity
327+
Today (0.2.0), the artifact has strong *deterministic* identity
328328
(sha256 hashes) but is not signed. The current artifact is suitable
329-
for use within a single trust boundary. This section will be revised
330-
when the Phase 1 work lands.
329+
for use within a single trust boundary. Cryptographic signing
330+
(Sigstore-style attestation) is deferred until artifacts need to
331+
cross trust boundaries; the post-Phase-D artifact track will ship
332+
`falsifyai verify` (integrity check + provenance display) and
333+
`falsifyai export --bundle` (productize the case-study extraction
334+
pattern) before signing is layered on.
331335

332336
---
333337

docs/RELEASE.md

Lines changed: 48 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -56,8 +56,42 @@ Run through every item before tagging. CI doesn't catch all of these.
5656
- `falsifyai/__init__.py` `__version__ = "..."`
5757
- `tests/unit/test_version.py` (update the hardcoded assertion)
5858

59-
7. **README badges + status line current.** The status line at the top
60-
should reflect the version being released.
59+
7. **Release-state metadata audit.** Several files contain a version
60+
string that means *"the current release"* — they MUST advance on
61+
every release. This is distinct from historical context references
62+
(e.g. *"unchanged from 0.1.0"*) which MUST NOT change. A blanket
63+
grep would produce both signals; the audit is a deliberate
64+
human-judgment pass, not an automated gate.
65+
66+
Drift-prone files to check on every release:
67+
- [ ] `README.md` status banner (top of file) — names the version
68+
and the shipped feature wave for the public.
69+
- [ ] `README.md` "Status and roadmap" section — `(current release)`
70+
marker matches the new version; previous version stays as
71+
historical entry.
72+
- [ ] `CITATION.cff``version:` and `date-released:` fields
73+
(machine-readable; consumed by citation tools).
74+
- [ ] `docs/EVIDENCE.md` — the *"Today (X.Y.Z), the artifact has…"*
75+
paragraph in §6.2; this is a release-state claim about current
76+
guarantees.
77+
- [ ] `docs/case-studies/*.md``pip install falsifyai==X.Y.Z`
78+
commands in reproduction sections (so readers install the
79+
version that contains the case study itself).
80+
- [ ] `CHANGELOG.md` — new `[X.Y.Z]: <release URL>` link added
81+
under the existing `[N-1.Y.Z]` reference at the bottom.
82+
- [ ] `docs/RELEASE.md` (this file) — concrete example tag
83+
commands and the "next dev marker" example should reference the
84+
most recent release as their illustrative case.
85+
86+
Files that *also* contain version strings but are historically
87+
accurate — do NOT change without specific reason:
88+
- `CHANGELOG.md` body lines mentioning prior versions in context
89+
(e.g. *"unchanged from 0.1.0"*).
90+
- `docs/case-studies/data/README.md` provenance — the FalsifyAI
91+
version recorded there reflects when the bundled artifacts were
92+
*generated*, not the current release.
93+
- `docs/ARCHITECTURE.md` discipline statements anchored to the
94+
0.1.0 baseline.
6195

6296
8. **Documentation links work.** Quick check that all
6397
`[link](docs/...)` references resolve on the branch you're tagging.
@@ -86,11 +120,11 @@ If `twine check` fails, the most common causes are:
86120
## Tag (triggers automated publish)
87121

88122
```bash
89-
# Create an annotated tag matching the version.
90-
git tag -a v0.1.0 -m "Release 0.1.0"
123+
# Create an annotated tag matching the version (substitute the new version below).
124+
git tag -a v0.2.0 -m "Release 0.2.0"
91125

92126
# Push the tag. This fires .github/workflows/publish.yml.
93-
git push origin v0.1.0
127+
git push origin v0.2.0
94128
```
95129

96130
The workflow:
@@ -150,8 +184,8 @@ falsifyai --help
150184

151185
1. Go to the [Releases page](https://github.com/ericckzhou/falsifyai/releases).
152186
2. Click "Draft a new release."
153-
3. Choose the tag you just pushed (`v0.1.0`).
154-
4. Title: `0.1.0 — Phase 0 MVP` (or equivalent thematic name).
187+
3. Choose the tag you just pushed (e.g. `v0.2.0`).
188+
4. Title: `<version> — <thematic name>` (e.g. `0.2.0 — Phase 1 first wave`).
155189
5. Body: copy the matching section from `CHANGELOG.md` verbatim.
156190
6. Publish.
157191

@@ -174,11 +208,13 @@ If the release warrants public attention:
174208
git push --force-with-lease origin dev
175209
```
176210

177-
2. **Bump version to next dev marker.** If this is the start of work
178-
toward 0.2.0:
179-
- `pyproject.toml`: `version = "0.2.0.dev0"`
180-
- `falsifyai/__init__.py`: `__version__ = "0.2.0.dev0"`
181-
- Open a PR titled `chore: bump version to 0.2.0.dev0`.
211+
2. **Bump version to next dev marker.** Optional but recommended when
212+
starting work toward the next release. For example, after `0.2.0`
213+
ships and the next planned release is `0.3.0`:
214+
- `pyproject.toml`: `version = "0.3.0.dev0"`
215+
- `falsifyai/__init__.py`: `__version__ = "0.3.0.dev0"`
216+
- `tests/unit/test_version.py`: update the asserted version string.
217+
- Open a PR titled `chore: bump version to <next>.dev0`.
182218

183219
3. **Open a `CHANGELOG.md` "## [Unreleased]" section** under the version
184220
header for the next release's entries.

docs/case-studies/01-invisible-character-substitution.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -181,7 +181,7 @@ Every command shown above runs against the bundled SQLite. No model calls, no AP
181181

182182
```bash
183183
# 1. Install FalsifyAI
184-
pip install falsifyai==0.1.0
184+
pip install falsifyai==0.2.0
185185

186186
# 2. Verify the bundle's integrity
187187
python -c "import hashlib; \
@@ -234,7 +234,7 @@ The U+202F substitution is memorable; the cross-model extraction brittleness is
234234

235235
## Reproduction checklist
236236

237-
- [ ] `pip install falsifyai==0.1.0`
237+
- [ ] `pip install falsifyai==0.2.0`
238238
- [ ] Clone this repository or download [`case-study-replays.db`](data/case-study-replays.db) (SHA256 above)
239239
- [ ] Run each command in Section 6
240240
- [ ] Confirm that every output matches what is shown in this document

0 commit comments

Comments
 (0)