For example apache httpd provides a directive MaxConnectionsPerChild which controls how frequently the server recycles processes by killing old ones and launching new ones.
What is the reason for killing the old threads at all after serving certain number of connections.
Doesn't this make the cpu cache cold unnecessarily?
From the docs:
Setting MaxConnectionsPerChild to a non-zero value limits the amount of memory that process can consume by (accidental) memory leakage.
So if you're leaking 1 MB per request (malloc() but not free()), you will gradually use up more and more memory until you run out and apache gets killed. But if you set MaxConnectionsPerChild 100, the child will gradually use up to 100 MB of memory, then get killed and go back down to 0.
The "hot cache" thing is applicable here, setting MaxConnectionsPerChild will slow down apache. That's why the default is infinite. MaxConnectionsPerChild is meant as inelegant duct-tape over memory leaks. A time-pressured programmer might prefer spending 1 minute setting MaxConnectionsPerChild, rather than 1 week hunting malloc() calls.