feat: add backfill support for llm-error-pages#2099
Conversation
|
This PR will trigger a minor release when merged. |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
duynguyen
left a comment
There was a problem hiding this comment.
Branch Review: llm-error-pages-backfill
Goal: Add backfill support for reprocessing historical weeks via auditContext.weekOffset, plus auto-process two weeks when run on Monday.
Potential Bug: Monday Week Offsets
handler.js:146-148
} else if (isMonday) {
weekOffsets = [-1, 0];
Offset 0 is the current week. On Monday, the current week is a few hours old at most — this will produce near-empty Athena results. This looks like it should be -2 and -1 (previous week + the week before), or just -1 and nothing more. Was running the in-progress current week intentional?
Breaking Change: auditResult Shape
runAuditAndSendToMystique now returns auditResult as an array ([{...}]) instead of a single object ({...}). This is persisted to the DB as the stored audit result.
- Step 2 (
submitForScraping) reads from a different step's result (step 1), so it's not affected. - No
opportunity-data-mapper.jsexists for this audit, so local code is fine. - Risk: External dashboards, monitoring, or reporting tools that query
auditResult.successon this audit type will silently break (they'll getundefinedsince arrays don't have.success).
This change deserves a callout in the PR description.
isMonday Hardcodes new Date()
handler.js:142
const isMonday = new Date().getUTCDay() === 1;
This works with sinon.useFakeTimers in the test, but it's fragile. The Monday test (handler.test.js:363-378) doesn't assert that generateReportingPeriods was called with [-1, 0] — it only checks output length (which is pre-determined by the mock). The test would pass even if the isMonday branch wasn't reached.
Tighten the assertion:
expect(mockGenerateReportingPeriods).to.have.been.calledWith(sinon.match.any, [-1, 0]);
derivePeriodFromBrokenLinks Fallback Not Directly Tested
guidance-handler.js:18-27
The null return path (no brokenLinks, or none with a matching suggestionId) falls back to:
generateReportingPeriods(new Date(), [-1]).weeks[0].periodIdentifier
The guidance-handler test now provides a suggestionId, so the regex always matches in tests. There's no coverage of the fallback. Consider adding a test case where brokenLinks is empty or has no suggestionId.
Positives
- Replacing
getWeekNumberwithisoCalendarWeekfrom shared-utils is the right call — fewer custom week calculations. periodIdentifierbaked into week objects (generateReportingPeriods) reduces duplication nicely.audit?.getId()optional chaining for the backfill path where no audit record exists yet is correct.- The
eslint-disable no-await-in-loopblock is appropriately scoped — serial Athena queries make sense given potential rate limits. - Default export routing pattern (
{ run(message, context) }) is compatible with index.js:305'stypeof handler.run === 'function'check.
|
Thanks @duynguyen for the review.
Added tests for 2,3 and 4. |
Please ensure your pull request adheres to the following guidelines:
Related Issues
LLMO-1678
Thanks for contributing!