Skip to content

Commit 53faa9b

Browse files
rmusser01rmusser01
authored
Phase 4: add OpenAPI raw response contracts (#1237)
* Phase 4: add OpenAPI raw response contracts * Address OpenAPI tag cache review feedback * Address remaining OpenAPI review feedback * Phase 4: cancel embeddings SSE producer on normal close * Fix audio transcription SRT response contract
1 parent 0731f62 commit 53faa9b

64 files changed

Lines changed: 2457 additions & 63 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎apps/extension/tests/unit/verify-openapi-client-paths.test.ts‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,4 +59,21 @@ describe("extractFallbackFieldNamesFromSource", () => {
5959
"final-field",
6060
])
6161
})
62+
63+
it("skips type annotation brackets before the fallback initializer", () => {
64+
const src = `
65+
export const MEDIA_ADD_SCHEMA_FALLBACK: Array<{
66+
name: string
67+
enum?: unknown[]
68+
}> = [
69+
{ name: "api_name" },
70+
{ name: "chunk_method", enum: ["semantic", "tokens"] }
71+
]
72+
`
73+
74+
expect(extractFallbackFieldNamesFromSource(src)).toEqual([
75+
"api_name",
76+
"chunk_method",
77+
])
78+
})
6279
})

‎backlog/tasks/task-13 - Address-PR-1237-embeddings-SSE-producer-shutdown-review.md‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
---
2+
id: TASK-13
3+
title: Address PR 1237 embeddings SSE producer shutdown review
4+
status: Done
5+
assignee: []
6+
created_date: '2026-05-03 20:16'
7+
updated_date: '2026-05-03 20:19'
8+
labels:
9+
- pr-review
10+
- openapi
11+
- embeddings
12+
- phase4
13+
dependencies: []
14+
references:
15+
- 'https://github.com/rmusser01/tldw_server/pull/1237'
16+
priority: medium
17+
---
18+
19+
## Description
20+
21+
<!-- SECTION:DESCRIPTION:BEGIN -->
22+
Resolve the non-inline CodeRabbit PR #1237 review-body finding that the unified embeddings orchestrator SSE normal shutdown path can await an infinite producer without cancelling it. Keep the change narrow to producer cancellation/lifecycle semantics and focused tests.
23+
<!-- SECTION:DESCRIPTION:END -->
24+
25+
## Acceptance Criteria
26+
<!-- AC:BEGIN -->
27+
- [x] #1 Unified embeddings orchestrator SSE normal generator close cancels the producer before awaiting it.
28+
- [x] #2 A focused regression fails before the fix and passes after it.
29+
- [x] #3 Focused tests, Bandit touched-source scope, and git diff --check are run and recorded.
30+
<!-- AC:END -->
31+
32+
## Implementation Plan
33+
34+
<!-- SECTION:PLAN:BEGIN -->
35+
1. Add a focused regression around STREAMS_UNIFIED orchestrator_events body iterator close that proves generator shutdown returns promptly and cancels the producer. 2. Patch only the normal shutdown path to cancel the producer before gather, matching the existing cancellation path. 3. Run focused embeddings SSE tests, Bandit on touched source/tests, and git diff --check. 4. Record verification and push the PR review-fix commit.
36+
<!-- SECTION:PLAN:END -->
37+
38+
## Implementation Notes
39+
40+
<!-- SECTION:NOTES:BEGIN -->
41+
RED: test_embeddings_orchestrator_events_unified_normal_close_cancels_producer timed out before the fix while awaiting the still-running producer task. GREEN: focused normal_close regression passed after the production change. Adjacent verification: test_orchestrator_sse_unified_flag.py plus test_orchestrator_sse.py passed with existing Redis-fixture skips (1 passed, 5 skipped); OpenAPI embeddings orchestrator contract selection passed (1 passed). Bandit source scope reported 0 findings in /tmp/bandit_pr1237_embeddings_sse_producer.json. git diff --check passed.
42+
<!-- SECTION:NOTES:END -->
43+
44+
## Final Summary
45+
46+
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
47+
Addressed the PR #1237 non-inline embeddings SSE review finding by cancelling the long-running unified orchestrator producer before awaiting it on normal stream completion. Added a focused regression proving the body iterator completes promptly and the producer task is cancelled.
48+
<!-- SECTION:FINAL_SUMMARY:END -->
49+
50+
## Definition of Done
51+
<!-- DOD:BEGIN -->
52+
- [x] #1 Acceptance criteria completed
53+
- [x] #2 Tests or verification recorded
54+
- [x] #3 Documentation updated when relevant
55+
- [x] #4 Bandit run for touched code when applicable or document non-code/environment skip
56+
- [x] #5 Final summary added
57+
- [x] #6 Known skips or blockers documented
58+
<!-- DOD:END -->

‎backlog/tasks/task-14 - Fix-PR-1237-audio-transcription-SRT-OpenAPI-contract.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
---
2+
id: TASK-14
3+
title: Fix PR 1237 audio transcription SRT OpenAPI contract
4+
status: Done
5+
assignee: []
6+
created_date: '2026-05-03 20:36'
7+
updated_date: '2026-05-03 20:39'
8+
labels:
9+
- openapi
10+
- audio
11+
- review-fix
12+
dependencies: []
13+
references:
14+
- 'https://github.com/rmusser01/tldw_server/pull/1237'
15+
priority: medium
16+
---
17+
18+
## Description
19+
20+
<!-- SECTION:DESCRIPTION:BEGIN -->
21+
Resolve the PR #1237 review finding where the audio transcription 200-response description mentions SRT but the OpenAPI 200-response content map does not expose an SRT-specific media type. The endpoint already supports response_format=srt, so align the generated contract and focused tests with runtime behavior.
22+
<!-- SECTION:DESCRIPTION:END -->
23+
24+
## Acceptance Criteria
25+
<!-- AC:BEGIN -->
26+
- [x] #1 Generated OpenAPI for /api/v1/audio/transcriptions and /api/v1/audio/translations documents an SRT media type when SRT is described as supported.
27+
- [x] #2 Focused contract tests/constants cover the SRT content type for both OpenAI-compatible audio routes.
28+
- [x] #3 Runtime SRT responses use the documented SRT media type or the description is adjusted to match actual runtime support.
29+
- [x] #4 Focused pytest verification and touched-scope Bandit checks are run and recorded.
30+
<!-- AC:END -->
31+
32+
## Implementation Plan
33+
34+
<!-- SECTION:PLAN:BEGIN -->
35+
1. Verify runtime SRT support and current OpenAPI response content. 2. Add failing contract/runtime assertions for SRT media type. 3. Update endpoint OpenAPI content and SRT response media type. 4. Run focused pytest, OpenAPI verifier, diff check, and Bandit.
36+
<!-- SECTION:PLAN:END -->
37+
38+
## Implementation Notes
39+
40+
<!-- SECTION:NOTES:BEGIN -->
41+
Red verification before implementation: focused pytest failed because generated OpenAPI lacked application/x-subrip and response_format=srt returned text/plain; charset=utf-8.
42+
43+
Implementation: added application/x-subrip to the shared audio transcript response content map and changed response_format=srt responses to return application/x-subrip. Updated OpenAPI contract constants and SRT runtime test coverage.
44+
45+
Verification: focused red run failed before implementation for missing application/x-subrip and text/plain SRT runtime response. Green checks: 73 related pytest tests passed, apps/packages/ui and apps/extension verify:openapi passed with existing reviewed exceptions, git diff --check passed, Bandit on audio_transcriptions.py reported zero findings.
46+
<!-- SECTION:NOTES:END -->
47+
48+
## Final Summary
49+
50+
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
51+
Changed the OpenAI-compatible audio transcription/translation response contract to include the SRT-specific application/x-subrip media type, matching documented response_format=srt support. The SRT runtime response now returns application/x-subrip instead of generic text/plain, and focused tests cover both generated OpenAPI content and actual SRT response headers. Verification covered the focused red-green regression, the broader OpenAPI/audio timed-segment pytest files, both frontend OpenAPI drift verifiers, git diff whitespace checks, and Bandit on the touched backend file.
52+
<!-- SECTION:FINAL_SUMMARY:END -->
53+
54+
## Definition of Done
55+
<!-- DOD:BEGIN -->
56+
- [x] #1 Acceptance criteria completed
57+
- [x] #2 Tests or verification recorded
58+
- [x] #3 Documentation updated when relevant
59+
- [x] #4 Bandit run for touched code when applicable or document non-code/environment skip
60+
- [x] #5 Final summary added
61+
- [x] #6 Known skips or blockers documented
62+
<!-- DOD:END -->

‎backlog/tasks/task-5 - Address-PR-1237-OpenAPI-tag-declaration-review-comment.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
---
2+
id: TASK-5
3+
title: Address PR 1237 OpenAPI tag declaration review comment
4+
status: Done
5+
assignee:
6+
- Codex
7+
created_date: '2026-05-03 18:32'
8+
updated_date: '2026-05-03 18:32'
9+
labels:
10+
- pr-review
11+
- openapi
12+
- phase4
13+
dependencies: []
14+
references:
15+
- 'https://github.com/rmusser01/tldw_server/pull/1237'
16+
- 'https://github.com/rmusser01/tldw_server/pull/1237#discussion_r3178558700'
17+
priority: medium
18+
---
19+
20+
## Description
21+
22+
<!-- SECTION:DESCRIPTION:BEGIN -->
23+
Resolve the live review thread on PR #1237 by verifying the OpenAPI tag declaration helper is not repeated after schema caching and making that behavior explicit without broadening the Phase 4 OpenAPI contract scope.
24+
<!-- SECTION:DESCRIPTION:END -->
25+
26+
## Acceptance Criteria
27+
<!-- AC:BEGIN -->
28+
- [x] #1 The unresolved review comment on tldw_Server_API/app/main.py is addressed with a narrowly scoped change or documented technical response.
29+
- [x] #2 Relevant OpenAPI contract tests verify that schema generation caching prevents repeated tag declaration work.
30+
- [x] #3 Focused backend tests and security checks for the touched scope are run and recorded.
31+
<!-- AC:END -->
32+
33+
## Implementation Plan
34+
35+
<!-- SECTION:PLAN:BEGIN -->
36+
1. Add a focused OpenAPI contract test that resets the app OpenAPI cache, wraps _ensure_openapi_operation_tags_declared, calls app.openapi() twice, and verifies the helper only runs during the first schema build.
37+
2. Add a concise comment in custom_openapi() documenting that tag declaration work is covered by app.openapi_schema caching.
38+
3. Run the focused OpenAPI contract test, git diff --check, and Bandit on the touched backend files. Record verification in the task.
39+
4. If verification passes, commit the narrow PR follow-up and push to codex/phase4-openapi-contract-testing; then re-check the review thread/status.
40+
<!-- SECTION:PLAN:END -->
41+
42+
## Implementation Notes
43+
44+
<!-- SECTION:NOTES:BEGIN -->
45+
Implemented a focused OpenAPI schema-cache regression test and added a cache-boundary comment in custom_openapi(). Verification: targeted pytest for the new regression plus existing tag-declaration contract passed (2 passed, 5 warnings); git diff --check passed; Bandit on main.py passed with zero findings; Bandit on touched app/test files passed with B101 skipped because pytest asserts are expected in tests. A full test_openapi_contracts.py run timed out in existing TestClient startup/teardown after one test, so the focused contract verification is the reliable signal for this review comment.
46+
<!-- SECTION:NOTES:END -->
47+
48+
## Final Summary
49+
50+
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
51+
Addressed PR #1237 review feedback by documenting that OpenAPI schema normalization is covered by app.openapi_schema caching and adding a regression test proving tag declaration normalization runs only during the first schema build. No production behavior change beyond the clarifying comment.
52+
<!-- SECTION:FINAL_SUMMARY:END -->
53+
54+
## Definition of Done
55+
<!-- DOD:BEGIN -->
56+
- [x] #1 Acceptance criteria completed
57+
- [x] #2 Tests or verification recorded
58+
- [x] #3 Documentation updated when relevant
59+
- [x] #4 Bandit run for touched code when applicable or document non-code/environment skip
60+
- [x] #5 Final summary added
61+
- [x] #6 Known skips or blockers documented
62+
<!-- DOD:END -->

‎backlog/tasks/task-6 - Address-remaining-PR-1237-OpenAPI-review-threads.md‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
---
2+
id: TASK-6
3+
title: Address remaining PR 1237 OpenAPI review threads
4+
status: Done
5+
assignee:
6+
- Codex
7+
created_date: '2026-05-03 18:37'
8+
updated_date: '2026-05-03 18:43'
9+
labels:
10+
- pr-review
11+
- openapi
12+
- phase4
13+
dependencies: []
14+
references:
15+
- 'https://github.com/rmusser01/tldw_server/pull/1237'
16+
- 'https://github.com/rmusser01/tldw_server/pull/1237#discussion_r3178567209'
17+
- 'https://github.com/rmusser01/tldw_server/pull/1237#discussion_r3178567210'
18+
- 'https://github.com/rmusser01/tldw_server/pull/1237#discussion_r3178567211'
19+
- 'https://github.com/rmusser01/tldw_server/pull/1237#discussion_r3178567212'
20+
- 'https://github.com/rmusser01/tldw_server/pull/1237#discussion_r3178567216'
21+
- 'https://github.com/rmusser01/tldw_server/pull/1237#discussion_r3178567219'
22+
- 'https://github.com/rmusser01/tldw_server/pull/1237#discussion_r3178567220'
23+
priority: medium
24+
---
25+
26+
## Description
27+
28+
<!-- SECTION:DESCRIPTION:BEGIN -->
29+
Resolve the additional CodeRabbit review threads on PR #1237 by verifying each OpenAPI contract/tagging finding against runtime code and applying narrow contract fixes where valid.
30+
<!-- SECTION:DESCRIPTION:END -->
31+
32+
## Acceptance Criteria
33+
<!-- AC:BEGIN -->
34+
- [x] #1 OpenAPI contracts for audio speech, chat document generation, quickstart fallback, HAL raw formats, and VN asset content match reachable runtime response media types.
35+
- [x] #2 OpenAPI tag normalization ignores malformed non-sequence tag values instead of expanding strings into single-character tags.
36+
- [x] #3 Public control-plane routes retain the health tag in the generated OpenAPI schema.
37+
- [x] #4 Focused contract tests, diff check, and Bandit verification are run and recorded.
38+
<!-- AC:END -->
39+
40+
## Implementation Plan
41+
42+
<!-- SECTION:PLAN:BEGIN -->
43+
1. Inspect each reviewed endpoint/helper against current runtime code and existing contract tests.
44+
2. Add or update focused OpenAPI contract tests that encode the valid reviewer findings before changing production code.
45+
3. Apply minimal response-contract/tagging fixes in the reviewed files.
46+
4. Run focused pytest coverage for the changed contract tests plus git diff --check and Bandit on touched backend files.
47+
5. Commit, push to codex/phase4-openapi-contract-testing, and resolve the addressed review threads.
48+
<!-- SECTION:PLAN:END -->
49+
50+
## Implementation Notes
51+
52+
<!-- SECTION:NOTES:BEGIN -->
53+
Implemented validated CodeRabbit follow-ups: generic PCM OpenAPI media type for audio speech and reading TTS, explicit chat document JSON/SSE response content, explicit quickstart HTML fallback response, HAL Atom/RSS media types, VN asset image media types, hardened operation tag extraction for malformed scalar tags, and restored health tags on public control-plane routes. Red test run before production edits failed 7 expected assertions; final focused run passed 12 tests with 19 warnings. git diff --check passed. Bandit on touched backend/test files passed with B101 skipped for pytest asserts and zero findings.
54+
<!-- SECTION:NOTES:END -->
55+
56+
## Final Summary
57+
58+
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
59+
Addressed the remaining PR #1237 OpenAPI review threads with narrow contract and tag fixes, plus focused regression coverage for the media-type and tagging gaps.
60+
<!-- SECTION:FINAL_SUMMARY:END -->
61+
62+
## Definition of Done
63+
<!-- DOD:BEGIN -->
64+
- [x] #1 Acceptance criteria completed
65+
- [x] #2 Tests or verification recorded
66+
- [x] #3 Documentation updated when relevant
67+
- [x] #4 Bandit run for touched code when applicable or document non-code/environment skip
68+
- [x] #5 Final summary added
69+
- [x] #6 Known skips or blockers documented
70+
<!-- DOD:END -->

‎tldw_Server_API/app/api/v1/endpoints/admin/admin_bundle_ops.py‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -370,7 +370,16 @@ async def get_bundle_metadata(
370370
# ---------------------------------------------------------------------------
371371
# Route 5: GET /backups/bundles/{bundle_id}/download
372372
# ---------------------------------------------------------------------------
373-
@router.get("/backups/bundles/{bundle_id}/download")
373+
@router.get(
374+
"/backups/bundles/{bundle_id}/download",
375+
response_class=FileResponse,
376+
responses={
377+
200: {
378+
"description": "Backup bundle zip file",
379+
"content": {"application/zip": {}},
380+
},
381+
},
382+
)
374383
async def download_bundle(
375384
bundle_id: str,
376385
request: Request,

‎tldw_Server_API/app/api/v1/endpoints/admin/admin_events_stream.py‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,16 @@ async def emit_admin_event(
122122
# SSE Endpoint
123123
# ---------------------------------------------------------------------------
124124

125-
@router.get("/events/stream")
125+
@router.get(
126+
"/events/stream",
127+
response_class=StreamingResponse,
128+
responses={
129+
200: {
130+
"description": "Server-sent events stream for admin dashboard events",
131+
"content": {"text/event-stream": {}},
132+
},
133+
},
134+
)
126135
async def admin_events_stream(
127136
request: Request,
128137
categories: str | None = Query(

‎tldw_Server_API/app/api/v1/endpoints/admin/admin_system.py‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -155,7 +155,18 @@ async def get_audit_log(
155155
)
156156

157157

158-
@router.get("/audit-log/export")
158+
@router.get(
159+
"/audit-log/export",
160+
responses={
161+
200: {
162+
"description": "Audit log export as JSON or CSV.",
163+
"content": {
164+
"application/json": {},
165+
"text/csv": {},
166+
},
167+
},
168+
},
169+
)
159170
async def export_audit_log(
160171
user_id: int | None = None,
161172
action: str | None = None,

‎tldw_Server_API/app/api/v1/endpoints/admin/admin_usage.py‎

Lines changed: 36 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,18 @@ async def run_usage_aggregate(day: str | None = Query(None, description="YYYY-MM
8282
) from exc
8383

8484

85-
@router.get("/usage/daily/export.csv", response_class=PlainTextResponse)
85+
@router.get(
86+
"/usage/daily/export.csv",
87+
response_class=PlainTextResponse,
88+
responses={
89+
200: {
90+
"description": "Daily usage export as CSV.",
91+
"content": {
92+
"text/csv": {},
93+
},
94+
},
95+
},
96+
)
8697
async def export_usage_daily_csv(
8798
user_id: int | None = None,
8899
start: str | None = Query(None, description="YYYY-MM-DD inclusive"),
@@ -111,7 +122,18 @@ async def export_usage_daily_csv(
111122
return resp
112123

113124

114-
@router.get("/usage/top/export.csv", response_class=PlainTextResponse)
125+
@router.get(
126+
"/usage/top/export.csv",
127+
response_class=PlainTextResponse,
128+
responses={
129+
200: {
130+
"description": "Top usage export as CSV.",
131+
"content": {
132+
"text/csv": {},
133+
},
134+
},
135+
},
136+
)
115137
async def export_usage_top_csv(
116138
start: str | None = Query(None, description="YYYY-MM-DD inclusive"),
117139
end: str | None = Query(None, description="YYYY-MM-DD inclusive"),
@@ -245,7 +267,18 @@ async def get_llm_top_spenders(
245267
)
246268

247269

248-
@router.get("/llm-usage/export.csv", response_class=PlainTextResponse)
270+
@router.get(
271+
"/llm-usage/export.csv",
272+
response_class=PlainTextResponse,
273+
responses={
274+
200: {
275+
"description": "LLM usage export as CSV.",
276+
"content": {
277+
"text/csv": {},
278+
},
279+
},
280+
},
281+
)
249282
async def export_llm_usage_csv(
250283
user_id: int | None = None,
251284
provider: str | None = None,

0 commit comments

Comments
 (0)