Make Deliver policy configurable#10
Merged
dhanashritidke11 merged 9 commits intofeature/zgfrom Oct 22, 2025
Merged
Conversation
Signed-off-by: Derek Wang <whynowy@gmail.com>
* add .gitlab-ci.yml * feat(EventSensor): Dead Letter Queue Trigger (argoproj#3199) - without Sensor dlq, a feature in v1.9 (cherry picked from commit 2451789) * comment out other builds (cherry picked from commit 17070e7) --------- Co-authored-by: aaronare <aaronare@zillowgroup.com>
Signed-off-by: Taleb Zeghmi <talebz@zillowgroup.com>
Add support for configuring JetStream deliver policy on a per-dependency basis in sensors. This allows users to control how messages are delivered from JetStream streams. Changes: - Add JetStreamConsumerConfig type with DeliverPolicy field to sensor API - Add JetStreamDeliverPolicy constants: all, last, new - Implement consumerOptionsForDependency() to apply deliver policy to NATS subscriptions - Add JetStream field to eventbus.Dependency struct - Generate deepcopy methods for new API types - Add comprehensive unit tests - Update documentation with examples - Add example sensor configuration demonstrating all policies Supported deliver policies (aligned with NATS JetStream): - all: Start from earliest available message in stream - last: Start with last message in stream - new: Start with messages created after consumer (default) Backward compatible: Defaults to new when not configured, preserving existing behavior.
This reverts commit 68b0874.
This reverts commit 06fddf7.
This reverts commit 5737687.
docs/concepts/sensor.md
Outdated
|
|
||
| When using JetStream as the EventBus, you can configure the deliver policy for each dependency using the `jetStream` field. This allows you to control how messages are delivered: | ||
|
|
||
| - **`all`**: Default policy. Start receiving from the earliest available message in the stream |
Collaborator
There was a problem hiding this comment.
from description.
Default behavior preserved: When not configured, defaults to "new" policy (same as before)
i think new is default from what i understand
Collaborator
Author
There was a problem hiding this comment.
Yes, you're correct. Currently, the argo-events code defaults to "new" since it is hardcoded in the project.
However, within Jetstream the default deliver policy is "all" which makes this documentation confusing.
Since we already have jetstream specific documentation in https://github.com/zillow/argo-events/pull/10/files#diff-4ab2c17d34a671056836f0920040e4f28863529225a12dcee63c8faf47e72ea6 I'll remove this section as it's too specific to jetstream to reside in Sensor
abdulazillow
approved these changes
Oct 10, 2025
dhanashritidke11
added a commit
that referenced
this pull request
Oct 28, 2025
…olicy-configurable Make Deliver policy configurable
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This MR adds support for configurable JetStream deliver policies for sensor dependencies. Previously, sensors always used a hardcoded deliver policy when subscribing to JetStream subjects. This change allows users to configure the deliver policy on a per-dependency basis, enabling control over which messages are delivered from JetStream streams.
What's New
Users can now configure the
deliverPolicyfield in sensor dependencies when using JetStream as the EventBus:Why is this needed?
The current consumer Deliver policy is hardcoded to DeliverNew. While this works well with the our existing DiscardOld+Limits eventbus config, it's not compatible with our proposed WorkQueue+DiscardNew policy and Sensor fails to connect to Eventbus with an error:

Supported Deliver Policies
Based on NATS JetStream documentation:
all: Start receiving from the earliest available message in the streamlast: Start with the last message added to the stream, or the last message matching the consumer's filter subject if definednew: Start receiving messages created after the consumer was created (currently hardcoded to this)Backward Compatibility
✅ Fully backward compatible - no breaking changes:
jetStreamconfiguration is completely optional"new"policy (same as before)jetStreamconfig continue to work exactly as beforeMigration Path
No migration needed. Users can opt-in to the new feature by adding the
jetStreamconfiguration to their sensor dependencies when desired.Test Results
Deliver Policy = All
Deliver Policy = New (Backward compatible)