Clear Stale Persistent Tasks in Stop/Pause API #1629
+904
−165
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mohit10011999
requested review from
ankitkala,
gbbafna,
krishna-ggk,
monusingh-1,
saikaranam-amazon and
soosinha
as code owners
ankitkala
reviewed
Jan 27, 2026
Member
ankitkala
left a comment
ankitkala
left a comment
There was a problem hiding this comment.
Two major feedback:
- The PR has lot of additional code changes which doesn't seems to be related to the actual change. Can you remove all the unnecessary changes so its easier to review
- Stale replication tasks are problem when you're trying to create the task again(start or resume). I think we should be able to simplify by just handling this during task creation here (need to verify though, with any stacktrace from last few occurence of the issue)
src/main/kotlin/org/opensearch/replication/action/index/TransportReplicateIndexAction.kt
Show resolved
Hide resolved
src/main/kotlin/org/opensearch/replication/action/index/TransportReplicateIndexAction.kt
Show resolved
Hide resolved
| /** | ||
| * Handles case where no replication state exists but stale artifacts might remain. | ||
| */ | ||
| private suspend fun handleMissingReplicationState(indexName: String) { |
Member
There was a problem hiding this comment.
Why do you want to handle the stale tasks during stop flow? shouldn't we only do this during start and resume?
Contributor
Author
There was a problem hiding this comment.
The aim is to prevent the stale tasks to occur on a best effort basis, so that STOP/PAUSE api can be called multiple times, making it idempotent and we execute the full workflow to cleanup:-
Goal: Remove/suspend replication artifacts
Stale artifacts: Leftovers from previous incomplete operations
User intent: "Make sure replication is stopped/paused"
Handling stale artifacts helps achieve the goal
src/main/kotlin/org/opensearch/replication/action/pause/TransportPauseIndexReplicationAction.kt
Show resolved
Hide resolved
mohit10011999
force-pushed
the
stalePersistentTasks
branch
17 times, most recently
from
96341a5 to
2351fa1
Compare
Signed-off-by: Mohit Kumar <[email protected]>
Signed-off-by: Mohit Kumar <[email protected]>
Signed-off-by: Mohit Kumar <[email protected]>
Signed-off-by: Mohit Kumar <[email protected]>
Signed-off-by: Mohit Kumar <[email protected]>
Signed-off-by: Mohit Kumar <[email protected]>
Signed-off-by: Mohit Kumar <[email protected]>
Signed-off-by: Mohit Kumar <[email protected]>
Signed-off-by: Mohit Kumar <[email protected]>
…cel" This reverts commit b85dc35. Signed-off-by: Mohit Kumar <[email protected]>
Signed-off-by: Mohit Kumar <[email protected]>
Signed-off-by: Mohit Kumar <[email protected]>
mohit10011999
force-pushed
the
stalePersistentTasks
branch
7 times, most recently
from
43629d5 to
e904ee1
Compare
This reverts commit 2351fa1. Signed-off-by: Mohit Kumar <[email protected]>
mohit10011999
force-pushed
the
stalePersistentTasks
branch
8 times, most recently
from
f407086 to
759e8dc
Compare
Signed-off-by: Mohit Kumar <[email protected]>
mohit10011999
force-pushed
the
stalePersistentTasks
branch
5 times, most recently
from
9a48b36 to
1cd2550
Compare
…narios Signed-off-by: Mohit Kumar <[email protected]>
mohit10011999
force-pushed
the
stalePersistentTasks
branch
from
1cd2550 to
2b60eb0
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
When CCR is stopped or paused, all the index and shard replication tasks should be stopped. But if the stop/ pause is not completely successful, some of the replication tasks might stay running. This can cause conflict when we restart/resume the replication.
We have taken below actions to rectify this:-
Related Issues
Resolves #[Issue number to be closed when this PR is merged]
Check List
--signoff.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.