Robustness against Deployments

[ThomasObenaus]

Sokar modifies the job specification upon scaling the job. To be concrete the count of the job is modified.
Also the CI/CD system modifies the job specification without knowing the current count of the job that was set by sokar.
This conflicting knowledge/ information on both systems leads to the result that the CI/CD system will overwrite the count that was set by sokar and thus reverts the scaling.

[ThomasObenaus]

Options

1. Deployment through Sokar

• Instead of directly talking to the nomad master the CI/CD system sends the deployment ticket to sokar, which then does the deployment

2. No count parameter in job-file

• Probably consul-template works for the count parameter as well
• Then it would be possible that the CI/CD system updates the job specification without alerting the count
• Sokar then has to scale by modifying the according key in Consul

3. Merge current count into Job Spec

• Instead of just overriding the job spec. during a deployment, the CI/CD system could just obtain the current count by getting the job spec from nomad. Then this number could just be replaced in the system.

4. Rescale directly if after external change

• Sokar could watch the current count of the job spec and compare it against its internal knowledge/ desired count.
• If it does not match the expectation, sokar could directly update the count.

[ThomasObenaus]

Option "4. Rescale directly if after external change" was implemented with #53