[fix](editlog) Add missing edit log roll check in bulk write path#62060
[fix](editlog) Add missing edit log roll check in bulk write path#62060dataroaring wants to merge 1 commit intoapache:masterfrom
Conversation
…ulk write path The logEdit(short op, List<T> entries) method increments txId by entries.size() but never checks whether txId has reached Config.edit_log_roll_num. This means bulk writes through this path (e.g. batch partition operations) never trigger edit log rolling, causing the current journal database to grow without bound. Both logEditDirectly() and flushEditLog() correctly check txId against edit_log_roll_num after incrementing. This fix adds the same check to the bulk write path for consistency.
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
There was a problem hiding this comment.
Pull request overview
Adds the missing edit-log rolling threshold check to the bulk logEdit(short op, List<T> entries) path so bulk writes also trigger rolling and prevent unbounded journal DB growth.
Changes:
- Add
Config.edit_log_roll_numthreshold check after incrementingtxIdin the bulk write path - Roll edit log and reset
txIdwhen the threshold is reached
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| LOG.info("txId {} is equal to or larger than edit_log_roll_num {}, will roll edit.", | ||
| txId, Config.edit_log_roll_num); | ||
| rollEditLog(); | ||
| txId = 0; |
There was a problem hiding this comment.
Resetting txId to 0 after rolling can drop any remainder when entries.size() pushes txId past the threshold by more than the exact roll amount (e.g., txId goes from 90 to 190 with roll=100). This can delay the next roll because the extra 90 transactions are effectively forgotten. Consider preserving the remainder (e.g., txId %= Config.edit_log_roll_num or txId -= Config.edit_log_roll_num in a loop if multiple thresholds can be crossed) so rolling behavior is consistent with the configured transaction count.
| txId = 0; | |
| txId %= Config.edit_log_roll_num; |
Summary
logEdit(short op, List<T> entries)incrementstxIdbyentries.size()but never checks whethertxIdhas reachedConfig.edit_log_roll_numlogEditDirectly()andflushEditLog()correctly check; this fix adds the same check to the bulk write pathTest plan
edit_log_roll_numtransactions via bulk write path🤖 Generated with Claude Code