lauantai 26. maaliskuuta 2016

Problem with Kubernetes SkyDNS healtz

I had some problems when trying to get DNS working on Kubernetes. I followed instructions from https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/dns. Everything seemed to be working well, but the pod got restarted after 30 seconds. The log for healthz -container had following entries:

2016/03/19 04:25:25 Client ip 10.0.96.1:50326 requesting /healthz probe servicing cmd sleep 10 && nslookup kubernetes.default.svc.kube.local localhost >/dev/null
2016/03/19 04:25:25 Healthz probe error: Result of last exec: nslookup: can't resolve 'kubernetes.default.svc.kube.local': Name does not resolve, at 2016-03-19 04:25:23.967737423 +0000 UTC, error exit status 1
After trying a lot of things, I found a bug report for Alpine Linux. Basically, the nslookup does not respect the server parameter, if /etc/resolv.conf  has entries. Comment on that issues recommends using dig or drill for querying.

So I made a simple image and pushed it into docker hub, https://hub.docker.com/r/jyrki/arm-kubernetes-healthz-drill/. Nothing fancy, just added drill (https://github.com/jyrkiput/arm-kubernetes-healthz-drill/blob/master/Dockerfile). I used the existing image as base as I wanted to have the exechealtz available.

Then I had to change the healtz command to
drill -q kubernetes.default.svc.kube.local @localhost

perjantai 18. maaliskuuta 2016

Kubernetes 1.2.0 beta-1 not starting on Raspbian 8.0

While trying to start kubernetes v1.2.0 on Raspbian 8.0 I ran into problems. Only k8s-master and k8s-master-proxy containers were started so the system was not getting up properly. Logs for k8s-master were telling following:

7215 kubelet.go:2365] skipping pod synchronization - [Failed to start ContainerManager system validation failed - Following Cgroup subsystem not mounted: [memory]]
Cgroup memory subsystem is not enabled by default. You can enable it by adding
cgroup_enable=memory
into /boot/cmdline. Reboot is needed after this.

You can check if the memory subsystem is enabled by listing /sys/fs/cgroup/ which should the have directory called "memory" among others
blkio  cpu  cpuacct  cpu,cpuacct  cpuset  devices  freezer  memory  net_cls  systemd