Odd heap usage pattern

I have a repeating process that:

1. gets some data from the database
2. builds some objects in memory, adding to a Collection
3. writes the data from the Collection to a file

All of the objects/Collections go out of scope or are set to null after each iteration. (The Collection is reused for each iteration.)

Using Java VisualVM, I see a graph that looks like this, which seems very odd considering that it's a repeating process. Yes, the data coming back from the database is different, but it's generally the same amount.

Why does the heap size decrease at first?

Why does the used heap get so close to the heap size in the middle?

(the ~30-second blip at 1:43 was just when VisualVM froze momentarily)

I'm not as big of an expert on GC as some are, but the general idea is that when you've started the program you've given it the initial heap size, max heap size and other relevant parameters and then it's go time.

However the GC has plenty of intelligence and different algorithms that are optimized for different kinds of tasks. A naive implementation would just keep the heap size static and then collect the garbage when it's full. That's known as a "stop the world" collection, because the collector needs to stop everything so it can perform a little (or big) clean up.

Modern GC doesn't just cause long pauses into running programs because it needs to clean up, so there's always a little clean up going on as seen from the sawtooth. But when you start a program the GC has no idea what the program is going to do and how it will use memory. Therefore it has to observe what's happening, analyze memory usage and then decide what amounts of memory it needs to keep available for immediate use, whether it needs to grow the current heap size or if it can decrease the current heap size.

Depending on the behaviour of your program and the GC algorithm being used you can see a lot of different patterns. As long as you're not experiencing linear growth that ends up in an OutOfMemoryError, you should be relatively safe. Unless of course you want to optimize what's happening to increase throughput, responsiveness etc., but that's a more advanced subject and is more relevant when you've gotten your code working the way you want it.