Conversation
Replaces the batched per-file write with a per-chunk upsert, so a crash mid-file no longer discards already-embedded chunks. On resume, each chunk's deterministic ID is checked via the new has_chunk() and skipped if present. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Persistent JSON-backed record at ~/.sentrysearch/dlq.json. Each entry stores the chunk ID, source file, start/end time, truncated error message, attempt count, and timestamp. Atomic writes via a .tmp rename. Not yet wired into the indexing pipeline — follow-up commit adds the retry wrapper that feeds this queue. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Wraps embed_video_chunk with exponential-backoff retry (default 3
attempts). Failures are classified:
- Permanent (OOM, missing file, decode error): routed to DLQ
immediately — retrying would just fail the same way.
- Transient: retried with 2/4/8s backoff before DLQ.
- Gemini quota/auth errors: bubble up so the user stops and fixes.
DLQ'd chunks are skipped on subsequent runs so one poisoned chunk
doesn't block the rest of the library. Summary line now reports DLQ
counts and points users at `--retry-failed` for re-attempts.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
sentrysearch index <dir> --retry-failed
Pulls entries back out of the DLQ and re-attempts them. Successful
retries remove the entry; failures get recorded again.
sentrysearch dlq list
Prints each failed chunk with source file, timestamp, attempt
count, and the recorded error.
sentrysearch dlq clear
Empties the DLQ (with confirmation).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Extracts the start/end-time math from chunk_video into a pure expected_chunk_spans() helper. The index command now probes the video's duration, computes the chunk IDs it would produce, and skips the file entirely if all of them are present in the store. Restores the fast path that was lost when is_indexed() was removed in favor of per-chunk resume. Files with skipped still-frame chunks won't hit the fast path (stored count < expected) and will fall through to the normal per-chunk loop, same as before. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
retry failed embeddings automatically and add DLQ