Troubleshooting omni-kube-service-exposer

[a-lohmann]

Hi all,

Every now and then I see the exposed services in Omni not working. The reason seems to be the omni-kube-service-exposer pod going in to CrashLoopBackOff. The only log line the pod is giving me is:

exec /kube-service-exposer: exec format error

It's probably a configuration error on my end, but how do I troubleshoot this? Is there a way to get some more details on the formatting error?

Tnx for the help!

[smira]

Are you arm64?

[a-lohmann]

Are you arm64?

No amd64.

[smira]

Either way, it looks like it pulled the wrong arch. There is a potential fix coming in the next version of Talos, as this issue is intermittent, and not reproducible.

[a-lohmann]

Great to hear there is a fix for this coming up. In the meanwhile this keeps occurring. Sometimes a reboot fixes it, sometimes it does not. I will check the arch next time I see it.

Is there a way to pull the right arch and fix this without rebooting? Tnx!

[smira]

Reboot should not fix it, or your problem is not the one I described.

With Talos 1.12 and below the only way is to re-provision the node.