Kubernetes services allow us to address a collection of pods through a stable endpoint. Depending on the service type, this endpoint may be addressable only from within the cluster, from the same network, or even from the public internet. When running on AWS, setting the service type for a service to LoadBalancer will create an AWS ELB, which can be used as the endpoint. Wih a Kubernetes cluster running on public subnets, this just works. However, if your Kubernetes cluster only runs on private subnets (which is an option when spinning up clusters with Kops), then this doesn’t work. It creates the service as expecting, but it gets stuck in a pending state, where it fails to create the ELB. Doing a
kubectl describe svc service-name -n namespace-name
shows the fact that it can’t find a subnet to create the load balancer. The fix for this is simple (once you know how!). You just need to tag one or more AWS public subnets with some special tags, which then allows Kubernetes to create the load balancer in one of those subnets. The tags in question depend on the version of Kubernetes. I’m using 1.6, and as such, added the following tags with empty values:
With these tags in place, I could create a service as follows:
kind: Service apiVersion: v1 metadata: name: hellopub spec: selector: app: hello-world ports: - protocol: "TCP" port: 80 targetPort: 5000 type: LoadBalancer
This created an ELB, and everything ran fine.
If you’re not on Kubernetes 1.6, then I think the following tags will work:
KubernetesCluster (value would be the name of your cluster) kubernetes.io/role/elb
This of course means that you can’t use the same subnet across clusters.