-
-
Notifications
You must be signed in to change notification settings - Fork 913
Description
Is your feature request related to a problem? Please describe.
I work with multiple queues with one asynq server, I have multiple queues and multiple groups, we can pass "GroupMaxSize" to the asynq server, but this applies to ALL the groups, I'd like to be able to set a max size per group or per queue, since some groups might make sense to have 500 max size and others 100 max size.
Describe the solution you'd like
I'd suggest an interface like so where we can follow the "Queues" options style by setting a "Groups"
and each group can have its own options, or if it is too complicated just pass an int for the max size?
srv := asynq.NewServerFromRedisClient(
tdb,
asynq.Config{
Concurrency: 10,
GroupAggregator: asynq.GroupAggregatorFunc(MyAggregate),
GroupMaxDelay: 2 * time.Minute, // used as default
GroupGracePeriod: 1 * time.Minute, // used as default
GroupMaxSize: 100, // used as default
Queues: map[string]int{
"queue_A": 6,
"queue_B": 2,
"queue_C": 2
},
// OPTION 1
Groups: map[string]GroupOptions{
"queue_A": GroupOptions{
GroupMaxDelay: 10 * time.Minute,
GroupGracePeriod: 2 * time.Minute,
GroupMaxSize: 1000,
},
"queue_B": GroupOptions{
GroupMaxSize: 300,
},
}
// OPTION 2
Groups: map[string]int{
"queue_A": 1000,
"queue_B": 300,
}
},
)
type GroupOptions struct {
GroupMaxDelay time.Duration
GroupGracePeriod time.Duration
GroupMaxSize int
}Describe alternatives you've considered
alternative solutions are to create multiple instances of the asynq server, but that have performance panelties and makes the code more complex.
Using a single MaxSize for all the groups, but that reduces my options to optimize bulk insertions and bulk processing.