Search Engine: Elastic

Article ID: 132716, created on Jun 1, 2018, last review on Jun 1, 2018

  • Applies to:
  • Operations Automation 7.3

Symptoms

During Kubernetes installation, kube-router pod fails to deploy.

The following messages could be found in /var/log/messages:

May 24 06:03:49.307 kubelet: I0524 06:03:49.302946   30217 kuberuntime_manager.go:757] checking backoff for container "kube-router" in pod "kube-router-jl4dv_kube-system(eb846bac-5f0e-11e8-9283-0050569d0362)"
May 24 06:03:49.820 kubelet: W0524 06:03:49.820150   30217 kubelet_node_status.go:1057] Failed to set some node status fields: can't get ip address of node gdpr. error: No default routes.

The logs of kube-router container contain:

# kubectl logs kube-router-29mbx -n kube-system
...
E0601 09:15:08.771227       1 reflector.go:205] github.com/cloudnativelabs/kube-router/vendor/k8s.io/client-go/informers/factory.go:73: Failed to list *v1.Node: Get https://10.96.0.1:443/api/v1/nodes?resourceVersion=0: dial tcp 10.96.0.1:443: connect: network is unreachable

GDPR deployment fails on the generate.py script:

2018-06-01 09:24:11.579 [INFO] POST https://localhost:6308//aps/2/applications ...
2018-06-01 09:24:17.222 [ERROR] API Error:
2018-06-01 09:24:17.222 [ERROR] {
        "error": "APS::Util::NetworkException",
        "message": "Network error: Net Exception: Network is unreachable.  Application endpoint: gdpr-backend.default.svc.cluster.local.  Request: POST /rest/application."
}

Cause

Missing default gateway configuration on the K8S server.

Resolution

Default gateway is required for Kubernetes deployment, as kube-router installation uses it to define the network to operate on.

Add the default gateway configuration, e.g.:

# ip route add default via 10.166.144.1
# echo "default via 10.166.144.1" >> /etc/sysconfig/network-scripts/route-eth0

Alternatively, it is possible to add the required route for Kubernetes network manually:

# ip route add 10.96.0.0/12 dev eth0 proto kernel scope link src 192.168.0.15
# echo "10.96.0.0/12 dev eth0 proto kernel scope link src 192.168.0.15" >> /etc/sysconfig/network-scripts/route-eth0

The issue was raised with Kubernetes community, but never addressed:

API Server not listening on cluster service IP when there's no default gateway

5356b422f65bdad1c3e9edca5d74a1ae 8fc71f07abe5b233fea1ae0377cd5e3d dd79f16c76b9dca75205ab5e9f5465ea 1941880841f714e458ae4dc3d9f3062d

Email subscription for changes to this article
Save as PDF