jm + fragmentation   1

Fix glibc's MALLOC_ARENA_MAX variable
It seems that recent versions of glibc (up to glibc 2.25 at least) have some dysfunctional behaviour around malloc's arenas on multi-CPU systems, massively inflating the number of arenas allocated, which inflate reported VM sizes and (for multi-threaded Ruby services in particular) fragmenting memory badly.

See also https://devcenter.heroku.com/articles/testing-cedar-14-memory-use

Presto issue reported with glibc malloc arena-per-thread behaviour resulting in Presto OOMs: https://github.com/prestodb/presto/issues/8993

Hadoop affected by the inflated VM sizes reported as a side effect: https://issues.apache.org/jira/browse/HADOOP-7154

Good detailed writeup from IBM's WebSphere blog: https://www.ibm.com/developerworks/community/blogs/kevgrig/entry/linux_glibc_2_10_rhel_6_malloc_may_show_excessive_virtual_memory_usage
ops  ruby  memory  malloc  allocation  arenas  tuning  fragmentation 
28 days ago by jm

Copy this bookmark:



description:


tags: