[DSIP-106][WorkerGroup] Persist All Worker Groups in the Database #17890
Description
Search before asking
- I had searched in the issues and found no similar feature requirement.
Description
The worker group represents a collection of workers, and every task must be associated with a worker group. However, the current worker group design has several usability and management issues.
Existing Problems
Worker group names are tightly coupled with permissions and tasks.
Worker group names are directly referenced in both permission control and task configuration. Once a worker group name is changed, existing permissions and tasks become invalid.
The default worker group is difficult to manage.
The default worker group implicitly contains all workers, and all tasks are allowed to use it. This makes resource isolation and governance nearly impossible.
A worker can only be dynamically bound to a single worker group via configuration.
This limits flexibility in resource organization and prevents a worker from serving multiple logical groups.
Proposal
This DSIP proposes improvements to the worker group mechanism:
All worker groups must have metadata persisted in the database.
Worker groups should become first-class entities with unified management, rather than being partially configuration-driven.
Workers can still dynamically register to worker groups via configuration, but the worker group must be pre-defined in the API server.
This ensures consistency and prevents accidental or inconsistent group creation.
A worker should be allowed to dynamically register to multiple worker groups.
This improves flexibility in resource sharing and logical grouping.
The default worker group should behave like a regular worker group and be empty by default.
Workers must explicitly join it, enabling proper governance and isolation.
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct