kubernetesopenshiftopenshift-originkubernetes-podulimit

Unable to Create many threads in OCP Containers


Recently we were getting the following exception in one of our Containers which was running a Java application in Openshift 4.2. This container used to run perfectly on Openshift 3.11.

Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:717

Within the containers, the ulimits looks perfectly fine. See the below image.

Ulimits

In Openshift 3.11 the same container could create 4096 Threads. But in Openshift 4.2 it just can create 1024 threads. Please see the Below Images,

OCP 3.11 Openshift 3.11

OCP 4.2 enter image description here

From the above ulimits it's evident that docker agent level configurations are done. Also, I have allocated enough memory for OS to create native threads. But I have no clue where this limit is set. How can I increase this global limit? Thanks in advance.


Solution

  • By default, OpenShift 3 uses docker as container runtime whereas OpenShift 4 uses cri-o as container runtime.

    According to Default pids_limit too low - by default there has been a limit to 1024 threads when using cri-o.