Advice on scaling strategy: 1 pod per transaction, each lasting 10–30s (with headroom)

[emanuelef]

Hi everyone,

I'm working on a use case where I want to autoscale a service with KEDA based on transactions per second (TPS), but with a twist:
Each transaction is handled by one pod and takes between 10 to 30 seconds to complete (median ~15s).
I want to maintain 1 pod per concurrent transaction, ideally with a small headroom (e.g., 20–30%) to absorb short bursts.
The transactions arrive via a RabbitMQ queue, so I'm evaluating the RabbitMQ scaler using either:
mode: QueueLength, or mode: MessageRate

My questions:

1. Is there a way to scale based on message rate or queue length but apply a scaling factor or "headroom" to have some more replicas than the exact number ? I know this value can be a float only when mode is MessageRate. So I cannot have 0.2 for QueueLength.
2. Is there a recommended approach to smooth the metric to avoid over-scaling or flapping, like using a moving average, custom scaler, or delayed cooldown?

If my goal is:

1 TPS → 5 pods
5 TPS → 25 pods
10 TPS → 50 pods
…

what’s the cleanest way to implement that logic?
I want to avoid running Prometheus if possible, and ideally keep things lightweight and cloud-agnostic (GKE, EKS, AKS).

Thanks for any guidance or best practices you can share!

[rickbrouwer]

Hi,

About the "headroom“. If you want something more than you get back you can use scaling modifiers. With these you can make sure that you add something to the value you get. See also:

https://keda.sh/docs/2.17/concepts/scaling-deployments/#scaling-modifiers

About flapping, you can use scaleUp and/or scaleDown behaviors and include a stabilizationWindowSeconds of for example 300 seconds to prevent rapid fluctuations (flapping) in the number of replicas.

so;

advanced:
    horizontalPodAutoscalerConfig:
      behavior:
        scaleDown:
          stabilizationWindowSeconds: 300

see also:

https://keda.sh/docs/2.17/reference/scaledobject-spec/

[emanuelef]

Thanks, I'll have a closer look to Scaling modifiers.
In the example I see there are two triggers so I wasn't sure I can use with only one.