inside the data nodes a number of different techniques for allocating memory is used.
there are 2 main variants
- page allocations
- object allocations (e.i slab allocator)
and for the object allocations there are a number of different variants
- RWPool, "ordinary" slab, only objects of same type on 1 page, free-list per page and object
- WOPool, allocations which are short-lived, no free-list just ref-count
- ArrayPool, fixed array of packed object
and I'm just now introducing a ArenaAllocator (which will be used for spj)
- RWPool, WOPool and ArenaAllocator/Pool are "new", meaning that they are build from start with support for more dynamic memory handling in mind.
- ArrayPool is very fixed, and exists both in the structured form, and various hard-coded variants.
one thing in common for all the "new" allocators is that they use magic numbers per object, so that every time a pointer (once per signal) that validity of the object is checked, and a trap is generated on a invalid memory access.
another thing in common is that they are all used by 1 thread at a time, and only pages are allocated/freed to a global free-list.
- to support variable length data, linked list of fixed size objects are used (DataBuffer)
- for input data (from e.g sockets) a special DataBuffer with additional thread-safety code is used so that data can be passed between threads wo/ copying.
but there is no malloc/free new/delete :-)
Sysbench: IO-bound and a fast server - In this post I share results for in-memory sysbench on a fast server using MyRocks, InnoDB and TokuDB.tl;dr MyRocks is more space efficient than InnoDB. In...
2 hours ago