Linux Certif

Toute la documentation sur la certification Linux LPI

erts_alloc.3erl

NAME

erts_alloc - An Erlang Run-Time System internal memory allocator library.

DESCRIPTION

erts_alloc is an Erlang Run-Time System internal memory allocator library. erts_alloc provides the Erlang Run-Time System with a number of memory allocators.

ALLOCATORS

Currently the following allocators are present:

 temp_alloc

Allocator used for temporary allocations.

 eheap_alloc

Allocator used for Erlang heap data, such as Erlang process heaps.

 binary_alloc

Allocator used for Erlang binary data.

 ets_alloc

Allocator used for ETS data.

 driver_alloc

Allocator used for driver data.

 sl_alloc

Allocator used for memory blocks that are expected to be
       short-lived.

 ll_alloc

Allocator used for memory blocks that are expected to be
       long-lived, for example Erlang code.

 fix_alloc

A very fast allocator used for some fix-sized
       data.  fix_alloc manages a set of memory pools from
       which memory blocks are handed out.  fix_alloc
       allocates memory pools from  ll_alloc. Memory pools
       that have been allocated are never deallocated.

 std_alloc

Allocator used for most memory blocks not allocated via any of
       the other allocators described above.

 sys_alloc

This is normally the default malloc implementation
       used on the specific OS.

 mseg_alloc

A memory segment allocator. mseg_alloc is used by other
       allocators for allocating memory segments and is currently only
       available on systems that have the  mmap system
       call. Memory segments that are deallocated are kept for a
       while in a segment cache before they are destroyed. When
       segments are allocated, cached segments are used if possible
       instead of creating new segments.  This in order to reduce
       the number of system calls made.

sys_alloc and fix_alloc are always enabled and cannot be disabled. mseg_alloc is always enabled if it is available and an allocator that uses it is enabled. All other allocators can be enabled or disabled. By default all allocators are enabled. When an allocator is disabled, sys_alloc is used instead of the disabled allocator.

The main idea with the erts_alloc library is to separate memory blocks that are used differently into different memory areas, and by this achieving less memory fragmentation. By putting less effort in finding a good fit for memory blocks that are frequently allocated than for those less frequently allocated, a performance gain can be achieved.

THE ALLOC_UTIL FRAMEWORK

Internally a framework called alloc_util is used for implementing allocators. sys_alloc, fix_alloc, and mseg_alloc do not use this framework; hence, the following does not apply to them.

An allocator manages multiple areas, called carriers, in which memory blocks are placed. A carrier is either placed in a separate memory segment (allocated via mseg_alloc) or in the heap segment (allocated via sys_alloc). Multiblock carriers are used for storage of several blocks. Singleblock carriers are used for storage of one block. Blocks that are larger than the value of the singleblock carrier threshold ( sbct) parameter are placed in singleblock carriers. Blocks smaller than the value of the sbct parameter are placed in multiblock carriers. Normally an allocator creates a "main multiblock carrier". Main multiblock carriers are never deallocated. The size of the main multiblock carrier is determined by the value of the mmbcs parameter.

Sizes of multiblock carriers allocated via mseg_alloc are decided based on the values of the largest multiblock carrier size ( lmbcs), the smallest multiblock carrier size ( smbcs), and the multiblock carrier growth stages ( mbcgs) parameters. If nc is the current number of multiblock carriers (the main multiblock carrier excluded) managed by an allocator, the size of the next mseg_alloc multiblock carrier allocated by this allocator will roughly be smbcs+nc*(lmbcs-smbcs)/mbcgs when nc <= mbcgs, and lmbcs when nc > mbcgs. If the value of the sbct parameter should be larger than the value of the lmbcs parameter, the allocator may have to create multiblock carriers that are larger than the value of the lmbcs parameter, though. Singleblock carriers allocated via mseg_alloc are sized to whole pages.

Sizes of carriers allocated via sys_alloc are decided based on the value of the sys_alloc carrier size ( ycs) parameter. The size of a carrier is the least number of multiples of the value of the ycs parameter that satisfies the request.

Coalescing of free blocks are always performed immediately. Boundary tags (headers and footers) in free blocks are used which makes the time complexity for coalescing constant.

The memory allocation strategy used for multiblock carriers by an allocator is configurable via the as parameter. Currently the following strategies are available:

Best fit

Strategy: Find the smallest block that satisfies the requested block size.

Implementation: A balanced binary search tree is used. The time complexity is proportional to log N, where N is the number of sizes of free blocks.

Address order best fit

Strategy: Find the smallest block that satisfies the requested block size. If multiple blocks are found, choose the one with the lowest address.

Implementation: A balanced binary search tree is used. The time complexity is proportional to log N, where N is the number of free blocks.

Good fit

Strategy: Try to find the best fit, but settle for the best fit found during a limited search.

Implementation: The implementation uses segregated free lists with a maximum block search depth (in each list) in order to find a good fit fast. When the maximum block search depth is small (by default 3) this implementation has a time complexity that is constant. The maximum block search depth is configurable via the mbsd parameter.

A fit

Strategy: Do not search for a fit, inspect only one free block to see if it satisfies the request. This strategy is only intended to be used for temporary allocations.

Implementation: Inspect the first block in a free-list. If it satisfies the request, it is used; otherwise, a new carrier is created. The implementation has a time complexity that is constant.

As of erts version 5.6.1 the emulator will refuse to use this strategy on other allocators than temp_alloc. This since it will only cause problems for other allocators.

SYSTEM FLAGS EFFECTING ERTS_ALLOC

Warning:

Only use these flags if you are absolutely sure what you are doing. Unsuitable settings may cause serious performance degradation and even a system crash at any time during operation.

Memory allocator system flags have the following syntax: +M<S><P> <V> where <S> is a letter identifying a subsystem, <P> is a parameter, and <V> is the value to use. The flags can be passed to the Erlang emulator ( erl) as command line arguments.

System flags effecting specific allocators have an upper-case letter as <S>. The following letters are used for the currently present allocators:

*

B: binary_alloc

*

D: std_alloc

*

E: ets_alloc

*

F: fix_alloc

*

H: eheap_alloc

*

L: ll_alloc

*

M: mseg_alloc

*

R: driver_alloc

*

S: sl_alloc

*

T: temp_alloc

*

Y: sys_alloc

The following flags are available for configuration of mseg_alloc:

 +MMamcbf <size>

       Absolute max cache bad fit (in kilobytes). A segment in the
       memory segment cache is not reused if its size exceeds the
       requested size with more than the value of this
       parameter. Default value is 4096. 

 +MMrmcbf <ratio>

       Relative max cache bad fit (in percent). A segment in the
       memory segment cache is not reused if its size exceeds the
       requested size with more than relative max cache bad fit
       percent of the requested size. Default value is 20.

 +MMmcs <amount>

       Max cached segments. The maximum number of memory segments
       stored in the memory segment cache. Valid range is
       0-30. Default value is 5.

 +MMcci <time>

       Cache check interval (in milliseconds). The memory segment
       cache is checked for segments to destroy at an interval
       determined by this parameter. Default value is 1000.

The following flags are available for configuration of fix_alloc:

 +MFe true

       Enable  fix_alloc. Note:  fix_alloc cannot be disabled.

The following flags are available for configuration of sys_alloc:

 +MYe true

       Enable  sys_alloc. Note:  sys_alloc cannot be disabled.

 +MYm libc

 malloc library to use. Currently only
       libc is available.  libc enables the standard
       libc malloc implementation. By default  libc is used.

 +MYtt <size>

       Trim threshold size (in kilobytes). This is the maximum amount
       of free memory at the top of the heap (allocated by
       sbrk) that will be kept by  malloc (not
       released to the operating system). When the amount of free
       memory at the top of the heap exceeds the trim threshold,
       malloc will release it (by calling
       sbrk). Trim threshold is given in kilobytes. Default
       trim threshold is 128.  Note: This flag will
       only have any effect when the emulator has been linked with
       the GNU C library, and uses its  malloc implementation.

 +MYtp <size>

       Top pad size (in kilobytes). This is the amount of extra
       memory that will be allocated by  malloc when
       sbrk is called to get more memory from the operating
       system. Default top pad size is 0.  Note: This flag
       will only have any effect when the emulator has been linked
       with the GNU C library, and uses its  malloc
       implementation.

The following flags are available for configuration of allocators based on alloc_util. If u is used as subsystem identifier (i.e., <S> = u) all allocators based on alloc_util will be effected. If B, D, E, H, L, R, S, or T is used as subsystem identifier, only the specific allocator identified will be effected:

 +M<S>as bf|aobf|gf|af

       Allocation strategy. Valid strategies are  bf (best fit),
       aobf (address order best fit),  gf (good fit),
       and  af (a fit). See 
       the description of allocation strategies in "the  alloc_util framework" section.

 +M<S>asbcst <size>

       Absolute singleblock carrier shrink threshold (in
       kilobytes). When a block located in an
       mseg_alloc singleblock carrier is shrunk, the carrier
       will be left unchanged if the amount of unused memory is less
       than this threshold; otherwise, the carrier will be shrunk.
       See also  rsbcst.

 +M<S>e true|false

       Enable allocator  <S>.

 +M<S>lmbcs <size>

       Largest ( mseg_alloc) multiblock carrier size (in
       kilobytes).  See  the description
       on how sizes for mseg_alloc multiblock carriers are decided
       in "the  alloc_util framework" section.

 +M<S>mbcgs <ratio>

       ( mseg_alloc) multiblock carrier growth stages. See
       the description on how sizes for
       mseg_alloc multiblock carriers are decided
       in "the  alloc_util framework" section.

 +M<S>mbsd <depth>

       Max block search depth. This flag has effect only if the
       good fit strategy has been selected for allocator
       <S>. When the good fit strategy is used, free
       blocks are placed in segregated free-lists. Each free list
       contains blocks of sizes in a specific range. The max block
       search depth sets a limit on the maximum number of blocks to
       inspect in a free list during a search for suitable block
       satisfying the request.

 +M<S>mmbcs <size>

       Main multiblock carrier size. Sets the size of the main
       multiblock carrier for allocator  <S>. The main
       multiblock carrier is allocated via  sys_alloc and is
       never deallocated.

 +M<S>mmmbc <amount>

       Max  mseg_alloc multiblock carriers. Maximum number of
       multiblock carriers allocated via  mseg_alloc by
       allocator  <S>. When this limit has been reached,
       new multiblock carriers will be allocated via
       sys_alloc.

 +M<S>mmsbc <amount>

       Max  mseg_alloc singleblock carriers. Maximum number of
       singleblock carriers allocated via  mseg_alloc by
       allocator  <S>. When this limit has been reached,
       new singleblock carriers will be allocated via
       sys_alloc.

 +M<S>ramv <bool>

       Realloc always moves. When enabled, reallocate operations will
       more or less be translated into an allocate, copy, free sequence.
       This often reduce memory fragmentation, but costs performance.
      

 +M<S>rmbcmt <ratio>

       Relative multiblock carrier move threshold (in percent). When
       a block located in a multiblock carrier is shrunk,
       the block will be moved if the ratio of the size of the returned
       memory compared to the previous size is more than this threshold;
       otherwise, the block will be shrunk at current location.

 +M<S>rsbcmt <ratio>

       Relative singleblock carrier move threshold (in percent). When
       a block located in a singleblock carrier is shrunk to
       a size smaller than the value of the
       sbct parameter,
       the block will be left unchanged in the singleblock carrier if
       the ratio of unused memory is less than this threshold;
       otherwise, it will be moved into a multiblock carrier. 

 +M<S>rsbcst <ratio>

       Relative singleblock carrier shrink threshold (in
       percent). When a block located in an  mseg_alloc
       singleblock carrier is shrunk, the carrier will be left
       unchanged if the ratio of unused memory is less than this
       threshold; otherwise, the carrier will be shrunk.
       See also  asbcst.

 +M<S>sbct <size>

       Singleblock carrier threshold. Blocks larger than this
       threshold will be placed in singleblock carriers. Blocks
       smaller than this threshold will be placed in multiblock
       carriers.

 +M<S>smbcs <size>

       Smallest ( mseg_alloc) multiblock carrier size (in
       kilobytes). See  the description
       on how sizes for mseg_alloc multiblock carriers are decided
       in "the  alloc_util framework" section.

 +M<S>t true|false|<amount>

       Multiple, thread specific instances of the allocator.
       This option will only have any effect on the runtime system
       with SMP support. Default behaviour on the runtime system with
       SMP support ( N equals the number of scheduler threads):
       

 temp_alloc

 N + 1 instances.

 ll_alloc

 1 instance.

Other allocators

 N instances when  N is less than or equal to          16. 16 instances when N is greater than
         16.

 temp_alloc will always use  N + 1 instances when
       this option has been enabled regardless of the amount passed.
       Other allocators will use the same amount of instances as the
       amount passed as long as it isn't greater than  N.       
      

Currently the following flags are available for configuration of alloc_util, i.e. all allocators based on alloc_util will be effected:

 +Muycs <size>

 sys_alloc carrier size. Carriers allocated via
       sys_alloc will be allocated in sizes which are
       multiples of the  sys_alloc carrier size. This is not
       true for main multiblock carriers and carriers allocated
       during a memory shortage, though.

 +Mummc <amount>

       Max  mseg_alloc carriers. Maximum number of carriers
       placed in separate memory segments. When this limit has been
       reached, new carriers will be placed in memory retrieved from
       sys_alloc.

Instrumentation flags:

 +Mim true|false

       A map over current allocations is kept by the emulator. The
       allocation map can be retrieved via the  instrument
       module.  +Mim true implies  +Mis true.
       +Mim true is the same as
       -instr.

 +Mis true|false

       Status over allocated memory is kept by the emulator. The
       allocation status can be retrieved via the  instrument
       module.

 +Mit X

       Reserved for future use. Do  not use this flag.

Note:

When instrumentation of the emulator is enabled, the emulator uses more memory and runs slower.

Other flags:

 +Mea min|max|r9c|r10b|r11b|config

 min

         Disables all allocators that can be disabled.
        

 max

         Enables all allocators (currently default).
        

 r9c|r10b|r11b

         Configures all allocators as they were configured in respective
         OTP release. These will eventually be removed.
        

 config

         Disables features that cannot be enabled while creating an
         allocator configuration with
         erts_alloc_config(3erl).
         Note, this option should only be used while running
         erts_alloc_config, not when using the created
         configuration.

        

Only some default values have been presented here. erlang:system_info(allocator), and erlang:system_info({allocator, Alloc}) can be used in order to obtain currently used settings and current status of the allocators.

Note:

Most of these flags are highly implementation dependent, and they may be changed or removed without prior notice.

erts_alloc is not obliged to strictly use the settings that have been passed to it (it may even ignore them).

erts_alloc_config(3erl) is a tool that can be used to aid creation of an erts_alloc configuration that is suitable for a limited number of runtime scenarios.

SEE ALSO

erts_alloc_config(3erl), erl(1), instrument(3erl), erlang(3erl)