I couldn't say for sure. I would advise sorting that top window by memory (shift+f, arrows to mem, press 's'). You could experiment with removing the Xms parameter.
Sometimes with larger heaps the garbage collector can be lazy, and then when it does run it has more work to do and a bigger pile of potential garbage to sift through, but I wouldn't expect that to cause what you're describing, as it also generally has more garbage it can collect, and it can meet its targets easily