VisualVM heap size doesn't follow used size

I'm using the VisualVM to profile a javafx 8 application that does some graphing and uses a lot more memory than I would like. It doesn't seem to be leaking, but for some reason my total heap never decreases even though my used heap spikes up and down as I select different files to graph. All the spikes up are when I graph a new file and then drops back down when I exit that screen, but total heap just shoots up and remains constant. Is this normal?

Yes that's normal. The JVM allocates more heap memory from the OS and doesn't give it back but the actual usage may vary, i.e. the currently unused portion of the heap may change.

One reason is that allocating memory from the OS is somewhat costly and the data might actually be fragmented once written. Garbage collection might free chunks of memory and thus the used size gets reduced but the still used chunks might be spread out all over the total heap memory.

I'm not sure how the JVM actually handles that in detail (in fact different JVMs might handle that differently) but you might look up free lists, buddy systems etc. that are used in such situations.

One could argue that the JVM could "defragment" the memory after garbage collection and release excess heap memory afterwards but that would be quite a performance hit, especially if lots of data would have to be moved around in RAM or even virtual/swap memory. As with many computing problems it's a tradeoff between space and cpu usage.