allocation   751

« earlier    

Fix glibc's MALLOC_ARENA_MAX variable
It seems that recent versions of glibc (up to glibc 2.25 at least) have some dysfunctional behaviour around malloc's arenas on multi-CPU systems, massively inflating the number of arenas allocated, which inflate reported VM sizes and (for multi-threaded Ruby services in particular) fragmenting memory badly.

See also

Presto issue reported with glibc malloc arena-per-thread behaviour resulting in Presto OOMs:

Hadoop affected by the inflated VM sizes reported as a side effect:

Good detailed writeup from IBM's WebSphere blog:
ops  ruby  memory  malloc  allocation  arenas  tuning  fragmentation 
march 2019 by jm
[1803.02796] Resource Polymorphism
Guillaume Munch-Maccagnoni

“We present a resource-management model for ML-style programming languages, designed to be compatible with the OCaml philosophy and runtime model. This is a proposal to extend the OCaml language with destructors, move semantics, and resource polymorphism, to improve its safety, efficiency, interoperability, and expressiveness. It builds on the ownership-and-borrowing models of systems programming languages (Cyclone, C++11, Rust) and on linear types in functional programming (Linear Lisp, Clean, Alms). It continues a synthesis of resources from systems programming and resources in linear logic initiated by Baker.

It is a combination of many known and some new ideas. On the novel side, it highlights the good mathematical structure of Stroustrup's "Resource acquisition is initialisation" (RAII) idiom for resource management based on destructors, a notion sometimes confused with finalizers, and builds on it a notion of resource polymorphism, inspired by polarisation in proof theory, that mixes C++'s RAII and a tracing garbage collector (GC).

The proposal targets a new spot in the design space, with an automatic and predictable resource-management model, at the same time based on lightweight and expressive language abstractions. It is backwards-compatible: current code is expected to run with the same performance, the new abstractions fully combine with the current ones, and it supports a resource-polymorphic extension of libraries. It does so with only a few additions to the runtime, and it integrates with the current GC implementation. It is also compatible with the upcoming multicore extension, and suggests that the Rust model for eliminating data-races applies.

Interesting questions arise for a safe and practical type system, many of which have already been thoroughly investigated in the languages and prototypes Cyclone, Rust, and Alms.”
PLT  programming-language  OCaml  research-language  memory  memory-management  allocation  garbage-collection  Rust  type-systems 
march 2019 by elliottcable

« earlier    

related tags

'no  2012  2018  affordable  alloca  allocate  allocator  analysis  and  ants  arc  arenas  array  arraypool  article  asset  at  attributes  autoboxing  base.allocation  basic.income  bearmarket  ben  blinding  blog  bonds  budget  buffer  bullmarket  business  c++  c  calculation  canada  capital  capitalism  carlson  catalog  celtic  cloud  code  committee  computing  consumption  continuations  corruption  count  cov  coverage  cpp  cybernetics  dashboard  data  dave  debugging  design  desktop  development  dirichlet  distributedcomputing  dividends  docker  dotnet  dropbox  dynamic  ecc  eco  economics  economist  economy  efis  escape-analysis  essay  etf  export  extents  file  finance  followup  founders  fragmentation  free  frostbite  fund  funding  funds  future  garbage-collection  garbagecollection  ge  gem  go  golang  govt  handles  haskell  have  header  heap  hfs  hierarchical  hinesburg  hn  hostplus  howto  http  hyper-v  hyperparameters  ibrox  immelt  important  income  increase  index  information  inline  integer  intention'  internet  investing  investment  ipv4  ipv6  jeff  jemalloc  journal  json  june  k8s  king:  kotlin  kube-eagle  kubernetes  language  large  latent  lda  leak  learning  library  licensing  lie  lifetime  light-licensing  link  llvm  logger  low-level  lstm  lte  mac  machinelearning  malloc  management  market  markets  memory-management  memory  memorymanagement  mix  mixed  ml  modeling  models  money  mssql  network  newlocal  newzealand  nlp  nz  object  ocaml  of  ofcom  openrefine  operating  ops  optimization  overflow  overview  paging  pass  pension  perf  performance  planned  plt  portfolio  pprof  prediction  problems  processing  product  profiling  programming-language  programming  prometheus  pst  python  r2  rails  ram  ramitsethi  rangers  rationing  recovery  reduce  register  reinforcement  research-language  research  reservation  resource  resources  retain  retirement  reversing  router  ruby  rubyonrails  rust  s4hana  scoped  secure  security  server  set  share  side  size  small  socialism  spectrum  spend  stack-based  stack  startup  static  stock  stocks  storage  stream  sunsuper  super  superannuation  swift  system  table  tax  taxes  text  ticket  tlb  tool  topic  topicmodels  transfer  tuning  type-systems  uk  valgrind  valuation  vanguard  veeam  visualisation  visualization  volume  wastewater  water  windows  wipe  zero 

Copy this bookmark: