c++gperftools

gperftools: Can the time interval for generating output files in the heap profiler of gperftools be configured?


My program experiences occasional memory spikes:

  1. The issue of memory consumption spiking occurs once every 5 to 7 runs.
  2. Each run takes a few minutes.

I would like to use the heap profiler of gperftools to analyze the issue now. Gperftools generates too many output files, almost creating one every second automatically. And my program needs to run for several minutes, that means too many output files. Is there a way to configure gperftools to generate a file every 10 seconds?

I have tried the env var of HEAP_PROFILE_ALLOCATION_INTERVAL and HEAP_PROFILE_TIME_INTERVAL, but it's not working. gperftools still generates a file every less than a second, but it adds some markings to the file generated "at the set time." like this:

Starting tracking the heap
...
Dumping heap profile to heap_profile.0006.heap (476 MB allocated cumulatively, 475 MB currently in use)
Dumping heap profile to heap_profile.0007.heap (5 sec since the last dump)
Dumping heap profile to heap_profile.0008.heap (653 MB allocated cumulatively, 651 MB currently in use)
...

here is my code:

class MyClass {
public:
    MyClass() {
        data = new int[1000];
    }
    ~MyClass() {
        delete[] data;
    }

private:
    int* data;
};

int main() {
    HeapProfilerStart("heap_profile");

    std::vector<MyClass*> objects;
    for (int i = 0; i < 3; ++i) {
        for (int i = 0; i < 100000; ++i) {
            objects.push_back(new MyClass());
        }
        std::this_thread::sleep_for(std::chrono::seconds(2));
    }

    HeapProfilerStop();

    std::cout << "Memory profiling finished!\n";
    return 0;
}

Solution

  • It should be possible to bump allocation interval sufficiently so that you get right frequency of profile dumps.

    But for your use-case I'd consider growth samples. Or capturing heap sample at the peak (you'd have to detect it manually though). See MallocExtension::GetHeapSample and MallocExtension::GetHeapGrowthStacks. You then dump those strings to file and feed it to pprof. And extra trick is to compare good versus bad runs with --base arg.