kubernetes.default.svc is not using KubePrism as endpoint even if KubePrism is enabled

[stevefan1999-personal]

I thought that kubernetes.default.svc should also be able to automatically use KubePrism, but it seems like it is still using the old endpoint specified in the machine config, is this an expected behavior?

[smira]

This is a Service resource managed by the kube-apiserver, it's not managed by Talos, and it can't use KubePrism, as it should work with CNI networking.

KubePrism will be used though by kube-proxy (or equivalent of it) to manage the actual iptables rules (or equivalent) for the Service.

[stevefan1999-personal]

However I do have Cilium connections established between my workers and control plane nodes, but I just cant access Kubernetes API within the workers. I configured the cluster endpoint of the workers to use https://127.0.0.1:7445.

[stevefan1999-personal]

What I said above still holds. Service kubernetes is managed by kube-apiserver, and the way Cilium itself with kube-apiserver depends on how you install it (if you follow Talos docs, it would use KubePrism). The way Service itself works on networking level is up to the way you configure Cilium.

But I see that the endpoint of kuberneres service is still using the internal NAT IP of the control plane

[smira]

You don't need to configure anything specifically besides enabling KubePrism in Talos. If enabled at bootstrap, that's all you need.

[stevefan1999-personal]

You don't need to configure anything specifically besides enabling KubePrism in Talos. If enabled at bootstrap, that's all you need.

Then what should I set for my control plane cluster endpoint IP to my workers? My workers only have public IP and doesn't have internal access to my home network behind NAT, so using the internal NAT IP obviously wont work for the workers to connect to the control plane, right?

[smira]

Right, but it has nothing to do with KubePrism. KubePrism provides internal load-balancer for machines to connect back to ocntrolplane. If they can't connect to the controlplane, KubePrism won't help of cource.

In your particular case enabling KubeSpan should fix the issue, and KubePrism will do its job correctly.