pythonmonitoringgoogle-cloud-runcpu-usagepsutil

Why does the psutil CPU tracker not work in Google Cloud Run?


I'm currently working on a Tracker of CPU & RAM in python using psutil. I've made some tries locally and everything appears to work fine, but, when I deploy to Google Cloud Run, the tracker is not able to track the usage of CPU. (The RAM's tracker still working.)

Why isn't the CPU tracker working, and how can I fix it?

In case it helps, Imma drop the code of my tracker below

import psutil
import os
import time

def process_memory():
    """
Use the psutil library to access the memory statistics of the process using the ID obtained with os.getpid(). 
The returned  value corresponds to the Resident Set Size (RSS), 
which is the portion of the process's memory stored in RAM.

    Returns:
        int: RAM memory used by the process (in bytes).
    """

    process = psutil.Process(os.getpid())
    mem_info = process.memory_info()
    return mem_info.rss

def Tracker(func):
    def wrapper(*args, **kwargs):
        
        #####################################
        ## -- Before Function Execution -- ##
        #####################################

        # Measure Memory 
        mem_before = process_memory()

        # Measure CPU percentage 
        psutil.cpu_percent(interval=None)
        time.sleep(1)  

        ##############################
        ## -- Function Execution -- ##
        ##############################
        responses, final_tracker = func(*args, **kwargs)

        ###########################
        ## -- After Execution -- ##
        ###########################

        # Measure memory 
        mem_after = process_memory()
        consumed_memory = (mem_after - mem_before) / (1024 ** 2)  # Convert bytes to MB
        
        # Measure CPU percentage
        cpu_usage = psutil.cpu_percent(interval=None)

        final_tracker = {
            "Memory Consumed (MB)": consumed_memory,
            "CPU Consumed (%)": cpu_usage
        }
 
        return responses, final_tracker
    return wrapper

I've tried to change the cloud run's configuration to set the CPU as always allocated, but that didn't work.


Solution

  • Procfile:

    web: python3 main.py
    

    requirements.txt:

    google-cloud-profiler==4.1.0
    psutil==6.0.0
    

    main.py:

    import googlecloudprofiler
    import psutil
    import os
    import time
    
    def process_memory():
        process = psutil.Process(os.getpid())
        mem_info = process.memory_info()
        return mem_info.rss
    
    def Tracker(func):
        def wrapper(*args, **kwargs):
            
            #####################################
            ## -- Before Function Execution -- ##
            #####################################
    
            # Measure Memory 
            mem_before = process_memory()
    
            # Measure CPU percentage 
            psutil.cpu_percent(interval=None)
            time.sleep(1)  
    
            ##############################
            ## -- Function Execution -- ##
            ##############################
            func(*args, **kwargs)
    
            ###########################
            ## -- After Execution -- ##
            ###########################
    
            # Measure memory 
            mem_after = process_memory()
            consumed_memory = (mem_after - mem_before) / (1024 ** 2)  # Convert bytes to MB
            
            # Measure CPU percentage
            cpu_usage = psutil.cpu_percent(interval=None)
    
            final_tracker = {
                "Memory Consumed (MB)": consumed_memory,
                "CPU Consumed (%)": cpu_usage
            }
     
            return final_tracker
        return wrapper
    
    @Tracker
    def add(x,y: int) -> int:
        return x + y
    
    if __name__ == "__main__":
        try:
            googlecloudprofiler.start(
                service="psutil",
                service_version="0.0.1",
                verbose=3,
            )
        except (ValueError, NotImplementedError) as exc:
            print(exc)
    
        print(add(4,6))
    

    When I run the code locally:

    {'Memory Consumed (MB)': 0.0, 'CPU Consumed (%)': 12.3}
    

    When I run the code locally under e.g. Podman:

    {'Memory Consumed (MB)': 0.0, 'CPU Consumed (%)': 15.5}
    

    When I run the code on Cloud Run without Cloud Profiler:

    {'Memory Consumed (MB)': 0.0, 'CPU Consumed (%)': 0.0}
    

    When I run the code on Cloud Run with Cloud Profiler:

    {'Memory Consumed (MB)': 2.375, 'CPU Consumed (%)': 4.0}