I've been experimenting with Kubernetes recently (an instance installed and configured via the Mantl.io project), and one of the more puzzling things that I wanted to figure out is how to use kubectl with a token. I'm still not quite sure why, but the built-in ServiceAccount token wasn't allowing me to authenticate using the public IP for the cluster. (Any suggestions are welcome!) Keep in mind, I already had kubectl and a kubeconfig file established from running minikube on my local workstation.
Here are the things that helped me figure out how to enable token authentication. I had to wade through a fair amount of Kubernetes documentation to sort how the system is configured. RTFD dude! Many clues revealed themselves during this digging.
- Kubernetes Authentication:
- This is where I started, while the documentation states "The signed JWT can be used as a bearer token to authenticate as the given service account. See above for how the token is included in a request. Normally these secrets are mounted into pods for in-cluster access to the API server, but can be used from outside the cluster as well." I had no success authenticating using the service account token.
- Accessing The Cluster:
- In this document, I used the troubleshooting information titled "Without kubectl proxy (post v1.3.x). This led me to realize that the ServiceAccount token was not even being used to authorize via localhost. In turn, I tried to determine how to update the "scope" for the ServiceAccount, but I could not quite determine how to configure that. If anyone knows this, would definitely appreciate a comment on how to configure that!
- Creating A Custom Cluster
- After scanning through this document, I felt like I might be more on the money...
- Clue #1: "You will run docker, kubelet, and kube-proxy outside of a container, the same way you would run any system daemon, so you just need the bare binaries. For etcd, kube-apiserver, kube-controller-manager, and kube-scheduler, we recommend that you run these as containers, so you need an image to be built." Ah ha, so somehow I can configure kube-apiserver inside of Kubernetes.
- Clue #2: "Your tokens and passwords need to be stored in a file for the apiserver to read. This guide uses /var/lib/kube-apiserver/known_tokens.csv. The format for this file is described in the authentication documentation." Okay...makes sense.
Clue #3: "While the basic node services (kubelet, kube-proxy, docker) are typically started and managed using traditional system administration/automation approaches, the remaining master components of Kubernetes are all configured and managed by Kubernetes:
their options are specified in a Pod spec (yaml or json) rather than an /etc/init.d file or systemd unit.
- they are kept running by Kubernetes rather than by init." Ah ha again...need to configure this inside Kubernetes some how.
- Clue #4: "The apiserver, controller manager, and scheduler will each run as a pod on the master node." Getting closer...
- Clue #5: "Apiserver pod template ... --token-auth-file=/srv/kubernetes/known_tokens.csv" Ok, so somehow I need to set that flag...
- Clue #6: "Place each completed pod template into the kubelet config dir (whatever --config= argument of kubelet is set to, typically /etc/kubernetes/manifests). The order does not matter: scheduler and controller manager will retry reaching the apiserver until it is up." And, bingo...gotta find the manifests directory on my system.
In sum, I needed to create the known_tokens.csv and put the right data in it. And, to add that, I needed to add that to Kubernetes itself, since the apiserver runs as a pod, and will automatically pick up the reconfiguration. Turns out that is in the manifests directory, and surprise, surprise, there is a "kube-apiserver.yaml" in my /etc/kubernetes/manifests.
Once I created the known_tokens.csv and added that configuration to the apiserver manifest, bingo, I was able to login using a token in my kubeconfig.