Cannot set thread priority to real time despite using cap_sys_nice

I have an application that checks on POSIX environment whether thread priorities can be set to real time by calling

struct sched_param param;
param.sched_priority = 1;
int canSetRealTimeThreadPriority = (pthread_setschedparam(pthread_self(), SCHED_FIFO, &param) == 0);

On one system system A this works, but on another system B the check fails and I would like to find out why.

On both systems:

• the application is started as a systemd service via a service startup script.

• calling getcap on the binary returns among others cap_sys_nice+eip.

• the service script defines that the application is run by a non root user via User=[non root user]

• the service scripts sets LimitRTPRIO=20

• calling sysctl -n kernel.sched_rt_runtime_us returns 950000, which should be the default

• calling sysctl -n kernel.sched_rt_period_usreturns 1000000, which should be the default

• systemctl show [serviceName] returns among others LimitRTPRIO=20

• calling the limits on the running process of the application (prlimit --pid [application_pid]) will show among others:

RESOURCE   DESCRIPTION                             SOFT      HARD UNITS
NICE       max nice prio allowed to raise             0         0
RTPRIO     max real-time priority                    20        20
RTTIME     timeout for real-time tasks        unlimited unlimited microsecs

On system B where it doesn't allow real time thread priorities:

• etc/security/limits.conf contains line

[non root user]    -    rtprio    20

• kernel version is 3.10.0-862.el7.x86_64 and OS version Red Hat Enterprise Linux Server release 7.4 (Maipo)

On system A where realtime thread priorities can be set:

• kernel version is 3.10.0-957.56.1.el7.x86_64 and OS version Red Hat Enterprise Linux Server release 7.6 (Maipo)

When I test on system A and remove cap_sys_nice+eip from the binary via setcap '' [binary] I also cannot set real time thread priorities. I assume some setting on system B overrides the cap_sys_nice setting because it has a higher priority, so I wonder what that can be.

It turned out the reason why realtime thread priorities could not be set system wide was another service running, which had following settings in service script defined:

CPUShares=20
CPUQuota=500%

After removing these settings from the script and restarting the service, thread priorities could be set system wide again.