springkubernetesrabbitmqkeda

KEDA Scaling Down pod efficiently (check useractivity before scaling down)


I have spring boot app that consume from rabbitmq I deployed this app on k8s and created keda file to scale it if the size of the queue pass x messages it work fine i got new pod created ,but now I want to make like a rule on the scale down part I want to make sure that there is no user using that pod before scale it down so i dont interpept any activity I searched on google and chatgpt I found something like prestop and readnessProbe ,Who Have an idea how this can be done I'll apreciate an kind of help :)


Solution

  • I assume that there is some limited time that your application need to process request like 45s from receiving request to complete.

    In such case I would use terminationGracePeriodSeconds which by default is 30s but you can extend it to any value.

    What is happening under the hood is when KEDA will start scaling down and your pod will be in Terminating state it is immediately removed from service endpoint and it is not receiving new requests (if service exist for that pod). Then it will send SIGTERM signal and wait until process will finish it's work (please be sure that your application handle SIGTERM properly). After processing current request pod should be killed before time defined by terminationGracePeriodSeconds. If processing of request take longer than terminationGracePeriodSeconds it will just kill pod without waiting and message should go back to queue.

    So if processing request is taking eg. 120s sample manifest should looks like this

    apiVersion: v1
    kind: Pod
    metadata:
      name: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
      terminationGracePeriodSeconds: 130