In order to make some robustness tests on a multi pod system, I need to simulate somehow a pod failure. I don't want to use scale down on the deployment since it is less realistic. Deleting pods is also not very functional in my case since pods are coming back in a short time.
I have checked the responses on the: How to simulate Power Failure In Kubernetes And don't match my needs. Any suggestion?
To simulate a network issue I would suggest to shutdown the one of the node or mess with it`s IP addresses so that it would become unreachable.
Another way would to simulate max out resources . You could exec into a pod and run something that will consume a lot of resources (CPU, RAM) while having set up limits for them. Alternatively you can set up limit below the actual usage.
You can also set up probes and simulate their failures. Failed Liveness probe would force pod to be restarted.
Lastly, you can also check Chaos Controller. It provides controller for chaos testing and supports irch set of failure scenarios.