Big JVM heaps can be Big Trouble
JVMs have never been good at dealing with large heaps. Large heaps mean lots of garbage collection work, and often means some pretty long stop-the-world GC pauses where nothing else can proceed. This can cause query/request timeouts, or even zookeeper session timeouts in SolrCloud mode.
Off-Heap Filters
Heliosearch/Solr has some pretty advanced filter caching, but it can take up a significant amount of memory, depending on the application. This is exactly the type of large, longer lived objects that can benefit by being moved off the JVM heap and explicitly managed. Off-heap memory is invisible to the garbage collector.
Heliosearch filters (Solr DocSet objects) are now allocated off-heap and reference counted so they can be freed as soon as they are no longer being used. The JVM GC no longer needs to waste time copying around these blocks of memory. This helps to both eliminate the long GC pauses as well as increase request throughput.
Read full article from Heliosearch/Solr Off-Heap Filters
No comments:
Post a Comment