Complete #1181: migration Job, kine containers and konnectivity-agent still have no securityContext knob (blocked under restricted PodSecurity)

[Jakob3xD]

Summary

#1181 added configurable (no-default) securityContext for the control-plane
containers and pod. Three other pod templates that the Kamaji controller creates
still have no securityContext and no way to set one, so they are
rejected under PodSecurity restricted:

1. the datastore-migration Job (management cluster)
2. the kine sidecar + kine chmod init container (management cluster,
   non-etcd datastores)
3. the konnectivity-agent DaemonSet/Deployment (tenant cluster)

Notably the konnectivity server already exposes
spec.addons.konnectivity.server.securityContext, but the agent does not —
an asymmetry worth closing.

What happened (migration Job)

On a management cluster whose Kamaji namespace enforces PodSecurity
restricted:latest, changing a TenantControlPlane's spec.dataStore triggers
the migration Job, which can never be scheduled:

Warning  FailedCreate  110s (x102 over 102m)  job-controller
  Error creating: pods "migrate-<uid>-xxxxx" is forbidden:
  violates PodSecurity "restricted:latest":
  allowPrivilegeEscalation != false (container "migrate" must set
    securityContext.allowPrivilegeEscalation=false),
  unrestricted capabilities (container "migrate" must set
    securityContext.capabilities.drop=["ALL"]),
  runAsNonRoot != true (pod or container "migrate" must set
    securityContext.runAsNonRoot=true),
  seccompProfile (pod or container "migrate" must set
    securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

Because the Job never reaches JobComplete, the reconcile keeps returning the
MigrationInProcessError sentinel
(internal/resources/datastore/datastore_migrate.go:167) and the TCP is pinned
in Migrating indefinitely — the tenant control plane is never repointed to the
new datastore.

The gaps (code references)

#	Workload	Where built	securityContext today	Cluster
1	Migration Job	internal/resources/datastore/datastore_migrate.go:106-137	none, no knob	management
2a	kine sidecar	internal/builders/controlplane/deployment.go:1012-1051	none, no knob	management
2b	kine chmod init	internal/builders/controlplane/deployment.go:947-973	none, no knob	management
3	konnectivity-agent	internal/resources/konnectivity/agent.go (pod ~219-245, container ~251-291); spec KonnectivityAgentSpec (api/v1alpha1/tenantcontrolplane_types.go:417-445) has no securityContext	none, no knob	tenant

For reference, what #1181 already covers (configurable, no default):

Workload	Field
apiserver / controller-manager / scheduler	spec.controlPlane.deployment.containerSecurityContexts.{apiServer,controllerManager,scheduler}
control-plane pod	spec.controlPlane.deployment.podSecurityContext
konnectivity server	spec.addons.konnectivity.server.securityContext (types.go:405)

Out of scope: CoreDNS and kube-proxy — their securityContext is rendered from
the decoded kubeadm addon manifests, not owned by a Kamaji field.

Proposed fix

Extend the #1181 pattern (configurable *corev1.SecurityContext /
*corev1.PodSecurityContext, nil ⇒ unset, no default) to the three remaining
workloads:

1. kine — add kine (and kineInit) to
   ControlPlaneContainerSecurityContexts, wired in buildKine, mirroring the
   existing apiserver/cm/scheduler handling.
2. konnectivity-agent — add securityContext to KonnectivityAgentSpec,
   mirroring the existing KonnectivityServerSpec.SecurityContext.
3. Migration Job — allow a securityContext on the Job's pod/container.

One open design question for the migration Job (input welcome): a plain
no-default field means the securityContext is only exercised during a rare,
easily-forgotten event (a datastore change) — an operator who forgets to set it
reproduces exactly this outage. Since the Job runs the Kamaji controller image
(already restricted-compliant in a restricted cluster), a lower-footgun
alternative is to have the migration Job inherit the controller's own
securityContext (with an optional override), rather than default to nil.
Happy to implement whichever the maintainers prefer.

Environment

• Kamaji: edge
• Datastore: etcd (kamaji-etcd), moved to a dedicated release; migration
  triggered by the datastore name change
• Management cluster namespace PodSecurity: restricted:latest (enforce)

I'm happy to open a PR for this.

[prometherion]

Since the Job runs the Kamaji controller image
(already restricted-compliant in a restricted cluster), a lower-footgun
alternative is to have the migration Job inherit the controller's own securityContext (with an optional override)

This sounds reasonable to me. Not sure how to provide an optional override, tho, besides not seeing the benefit.