ibm-cloud-private

IBM Cloud Private 2.1.0.3 unable to access port 8443 after installation


I am currently facing this issue on ICP 2.1.0.3 where after an installation, with all the pods are up and running but the port 8443 is not listening and the platform-ui container seems to be having issue connecting to 10.0.0.25, which is the ClusterIP service for icp-management-ingress. The ICP was installed on fresh VMs where both iptables and ufw are inactive.

Below is the log of the container after a restart.

root@icpmaster:/opt/ibm-cloud-private-ce-2.1.0.3/cluster# docker logs cbaaebabd4c9
[2019-03-06T04:12:38.429] [INFO] [platform-ui] [server] [pid 1] [env production] started.
[HPM] Proxy created: /  ->  https://icp-management-ingress:8443
[HPM] Proxy rewrite rule created: "^/catalog/api/proxy" ~> ""
[2019-03-06T04:13:33.455] [INFO] [platform-ui] [server] Starting express server.
[2019-03-06T04:13:33.653] [INFO] [platform-ui] [server] Platform UI listening on http port 3000.
[2019-03-06T04:13:46.373] [ERROR] [platform-ui] [service-watcher] Error making request: Error: connect ECONNREFUSED 10.0.0.25:8443
GET https://icp-management-ingress:8443/kubernetes/api/v1/services?labelSelector=inmenu%3Dtrue HTTP/1.1
Accept: application/json
Authorization: Bearer ***

Error: connect ECONNREFUSED 10.0.0.25:8443

[Edit] So I have restarted the kubelet and docker. Then found this in the kubelet's log related to cgroups about file or directory not found, I wonder is it related to Docker, however my Docker is within the supported version 17.12.1-ce

kubelet's log (trimmed the log)

-- Logs begin at Tue 2019-03-05 22:52:15 +08, end at Thu 2019-03-07 21:36:31 +08. --
Error while processing event ("/sys/fs/cgroup/cpu,cpuacct/system.slice/var-lib-docker-overlay2- xxx -merged.mount: no such file or directory
Error while processing event ("/sys/fs/cgroup/blkio/system.slice/var-lib-docker-overlay2- xxx -merged.mount": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/blkio/system.slice/var-lib-docker-overlay2- xxx -merged.mount: no such file or directory
Error while processing event ("/sys/fs/cgroup/memory/system.slice/var-lib-docker-overlay2- xxx -merged.mount": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/memory/system.slice/var-lib-docker-overlay2- xxx-merged.mount: no such file or directory  Error while processing event ("/sys/fs/cgroup/devices/system.slice/var-lib-docker-overlay2- xxx -merged.mount: no such file or directory
Error while processing event ("/sys/fs/cgroup/cpu,cpuacct/system.slice/var-lib-docker-containers- xxx -shm.mount: no such file or directory
Error while processing event ("/sys/fs/cgroup/blkio/system.slice/var-lib-docker-containers- xxx -shm.mount: no such file or directory
Error while processing event ("/sys/fs/cgroup/memory/system.slice/var-lib-docker-containers- xxx -shm.mount": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/memory/system.slice/var-lib-docker-containers- xxx-shm.mount: no such file or directory

Status of other pods

root@icpmaster:/opt# kubectl --kubeconfig=/var/lib/kubelet/kubelet-config get pods -n kube-system -o wide

NAME                                                  READY     STATUS    RESTARTS   AGE       IP            NODE
auth-apikeys-tj57w                                    1/1       Running   2          1d        10.1.9.5      10.113.64.6
auth-idp-c84cb                                        2/3       Running   15         1d        10.1.9.8      10.113.64.6
auth-pap-28tnl                                        1/1       Running   2          1d        10.1.9.32     10.113.64.6
auth-pdp-bwn9k                                        1/1       Running   2          1d        10.1.9.30     10.113.64.6
calico-kube-controllers-759f7fc556-bfnn8              1/1       Running   0          57m       10.113.64.6   10.113.64.6
calico-node-bdnrc                                     2/2       Running   46         1d        10.113.64.6   10.113.64.6
calico-node-h8jnd                                     2/2       Running   4          1d        10.113.64.8   10.113.64.8
catalog-ui-7ctqv                                      1/1       Running   2          1d        10.1.9.14     10.113.64.6
default-backend-7c6d6df9d5-j4pl9                      1/1       Running   0          57m       10.1.9.22     10.113.64.6
heapster-5649f84695-vfjjw                             2/2       Running   0          1h        10.1.9.4      10.113.64.6
helm-api-76c8d8bc7-8qjxf                              2/2       Running   3          57m       10.1.9.2      10.113.64.6
helm-repo-7455d96-bg2td                               1/1       Running   0          58m       10.1.9.19     10.113.64.6
icp-management-ingress-xgp95                          1/1       Running   3          1d        10.1.9.62     10.113.64.6
icp-mongodb-0                                         1/1       Running   23         1d        10.1.9.10     10.113.64.6
image-manager-0                                       2/2       Running   6          1d        10.113.64.6   10.113.64.6
k8s-etcd-10.113.64.6                                  1/1       Running   2          1d        10.113.64.6   10.113.64.6
k8s-master-10.113.64.6                                3/3       Running   6          1d        10.113.64.6   10.113.64.6
k8s-proxy-10.113.64.6                                 1/1       Running   2          1d        10.113.64.6   10.113.64.6
k8s-proxy-10.113.64.8                                 1/1       Running   3          1d        10.113.64.8   10.113.64.8
kube-dns-ltdb4                                        3/3       Running   37         1d        10.1.9.34     10.113.64.6
logging-elk-client-65745dcd68-b69wb                   2/2       Running   0          1h        10.1.9.44     10.113.64.6
logging-elk-data-0                                    1/1       Running   0          56m       10.1.9.16     10.113.64.6
logging-elk-filebeat-ds-7cb78                         1/1       Running   2          1d        10.1.214.67   10.113.64.8
logging-elk-filebeat-ds-vmfbk                         1/1       Running   2          1d        10.1.9.36     10.113.64.6
logging-elk-logstash-76c548744b-n24c5                 1/1       Running   0          1h        10.1.9.17     10.113.64.6
logging-elk-master-686fbdd984-kpt7s                   1/1       Running   0          1h        10.1.9.56     10.113.64.6
mariadb-0                                             1/1       Running   8          1d        10.113.64.6   10.113.64.6
metrics-server-7f4fdb695f-7rsd5                       1/1       Running   7          1h        10.1.9.20     10.113.64.6
nginx-ingress-controller-gjnnb                        1/1       Running   9          1d        10.1.9.9      10.113.64.6
platform-api-dq4p8                                    1/1       Running   2          1d        10.1.9.7      10.113.64.6
platform-deploy-6kzds                                 1/1       Running   2          1d        10.1.9.11     10.113.64.6
platform-ui-6kzzn                                     0/1       Running   46         1d        10.1.9.35     10.113.64.6
rescheduler-g85d5                                     1/1       Running   2          1d        10.113.64.6   10.113.64.6
service-catalog-apiserver-rlfj5                       1/1       Running   6          1d        10.1.9.28     10.113.64.6
service-catalog-controller-manager-5b654dc8b8-jfj64   1/1       Running   5          57m       10.1.9.15     10.113.64.6
tiller-deploy-c59888d97-l7rhk                         1/1       Running   3          57m       10.113.64.6   10.113.64.6
unified-router-zmxwh                                  1/1       Running   2          1d        10.1.9.24     10.113.64.6
update-secrets-cpp6j                                  1/1       Running   10         57m       10.1.9.18     10.113.64.6

Solution

  • auth-idp-c84cb pod is not fully running (2/3 instead of 3/3) which will cause platform-ui container to have issue connecting to 10.0.0.25:8443. This is why the platform-ui pod is not fully running either (platform-ui-6kzzn 0/1).

    The cause for auth-idp pods not fully starting is often due to the ICP cluster/environment not having sufficient resources. Please re-install your ICP cluster after increasing the resources to meet these hardware requirements https://www.ibm.com/support/knowledgecenter/SSBS6K_2.1.0.3/supported_system_config/hardware_reqs.html

    Especially pay attention to the number of CPUs, RAM, and disk space.