Kubernetes Recipe - Basic setup not working

BenjiReis

Did the manual change solve the issue?

ralphsmeets

@BenjiReis
No, the manual change didn't solve the problem

BenjiReis

Did you make sure /proc/sys/net/bridge/bridge-nf-call-iptables is set to 1?

Our implementation uses kube-router wich requirest this setting.

ralphsmeets

@BenjiReis said in Kubernetes Recipe - Basic setup not working:

/proc/sys/net/bridge/bridge-nf-call-iptables

Just checked:

debian@master:~$ more /proc/sys/net/bridge/bridge-nf-call-iptables
1

BenjiReis

Ok thanks, then I don't understant why there's a pb unfortunately...

The recipe follow this doc to create the cluster: https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/

@suaro how does your manual install differs from the doc used by the recipe? Perhaps we can dig there.

BenjiReis

And the pod network is made with kube-router: https://github.com/cloudnativelabs/kube-router/blob/master/docs/kubeadm.md

ralphsmeets

I'm going to have a look into it!
Thanks and hopefully I'll find the cullpritt, so the recipe can be updated with some nice new ingrediënts

BenjiReis

Thanks

I'll try to investigate myself as well when i can, do not hesitate to come back here if you find anything.

ralphsmeets

I got it working. Seems like the podCIDR wasn't set. Setting it manually by patching the nodes worked for me:

for node in master node-1 node-2 node-3 do 
  kubectl patch node $node -p '{"spec":{"podCIDR":"10.96.0.0/12"}}'
done

Not sure if this is a problem with the recipe, or if this is a bug in kube-router/kube-controller-manager. Anyway, If have my cluster up and running now!!!

ralphsmeets

It seems that Debian Buster has some problems with Kubernetes. While this base setup is working, one should also assure that every tool uses the legacy iptables. If not, pod's will not be able to reach the kubernetes api... And then... failure all over!
So we also need:

update-alternatives --set iptables /usr/sbin/iptables-legacy
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
update-alternatives --set arptables /usr/sbin/arptables-legacy
update-alternatives --set ebtables /usr/sbin/ebtables-legacy

```