Class AdaptivePoolingAllocator
- java.lang.Object
-
- io.netty.buffer.AdaptivePoolingAllocator
-
- All Implemented Interfaces:
AdaptiveByteBufAllocator.AdaptiveAllocatorApi
@UnstableApi final class AdaptivePoolingAllocator extends java.lang.Object implements AdaptiveByteBufAllocator.AdaptiveAllocatorApi
An auto-tuning pooling allocator, that follows an anti-generational hypothesis.The allocator is organized into a list of Magazines, and each magazine has a chunk-buffer that they allocate buffers from.
The magazines hold the mutexes that ensure the thread-safety of the allocator, and each thread picks a magazine based on the id of the thread. This spreads the contention of multi-threaded access across the magazines. If contention is detected above a certain threshold, the number of magazines are increased in response to the contention.
The magazines maintain histograms of the sizes of the allocations they do. The histograms are used to compute the preferred chunk size. The preferred chunk size is one that is big enough to service 10 allocations of the 99-percentile size. This way, the chunk size is adapted to the allocation patterns.
Computing the preferred chunk size is a somewhat expensive operation. Therefore, the frequency with which this is done, is also adapted to the allocation pattern. If a newly computed preferred chunk is the same as the previous preferred chunk size, then the frequency is reduced. Otherwise, the frequency is increased.
This allows the allocator to quickly respond to changes in the application workload, without suffering undue overhead from maintaining its statistics.
Since magazines are "relatively thread-local", the allocator has a central queue that allow excess chunks from any magazine, to be shared with other magazines. The
createSharedChunkQueue()
method can be overridden to customize this queue.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description (package private) static class
AdaptivePoolingAllocator.AdaptiveByteBuf
private static class
AdaptivePoolingAllocator.AllocationStatistics
private static class
AdaptivePoolingAllocator.Chunk
(package private) static interface
AdaptivePoolingAllocator.ChunkAllocator
The strategy for howAdaptivePoolingAllocator
should allocate chunk buffers.private static class
AdaptivePoolingAllocator.Magazine
(package private) static class
AdaptivePoolingAllocator.MagazineCaching
-
Field Summary
Fields Modifier and Type Field Description private static int
BUFS_PER_CHUNK
private static int
CENTRAL_QUEUE_CAPACITY
The capacity if the central queue that allow chunks to be shared across magazines.private java.util.Queue<AdaptivePoolingAllocator.Chunk>
centralQueue
private AdaptivePoolingAllocator.ChunkAllocator
chunkAllocator
private static int
EXPANSION_ATTEMPTS
private boolean
freed
private static int
INITIAL_MAGAZINES
private java.util.Set<AdaptivePoolingAllocator.Magazine>
liveCachedMagazines
private static int
MAGAZINE_BUFFER_QUEUE_CAPACITY
The capacity if the magazine local buffer queue.private java.util.concurrent.locks.StampedLock
magazineExpandLock
private AdaptivePoolingAllocator.Magazine[]
magazines
private static int
MAX_CHUNK_SIZE
The maximum size of a pooled chunk, in bytes.private static int
MAX_STRIPES
private static int
MIN_CHUNK_SIZE
private static java.lang.Object
NO_MAGAZINE
private static int
RETIRE_CAPACITY
private FastThreadLocal<java.lang.Object>
threadLocalMagazine
-
Constructor Summary
Constructors Constructor Description AdaptivePoolingAllocator(AdaptivePoolingAllocator.ChunkAllocator chunkAllocator, AdaptivePoolingAllocator.MagazineCaching magazineCaching)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description ByteBuf
allocate(int size, int maxCapacity)
(package private) void
allocate(int size, int maxCapacity, AdaptivePoolingAllocator.AdaptiveByteBuf into)
Allocate into the given buffer.private AdaptivePoolingAllocator.AdaptiveByteBuf
allocate(int size, int maxCapacity, java.lang.Thread currentThread, AdaptivePoolingAllocator.AdaptiveByteBuf buf)
private AdaptivePoolingAllocator.AdaptiveByteBuf
allocateFallback(int size, int maxCapacity, java.lang.Thread currentThread, AdaptivePoolingAllocator.AdaptiveByteBuf buf)
private static java.util.Queue<AdaptivePoolingAllocator.Chunk>
createSharedChunkQueue()
Create a thread-safe multi-producer, multi-consumer queue to hold chunks that spill over from the internal Magazines.protected void
finalize()
private void
free()
private AdaptivePoolingAllocator.Magazine
getFallbackMagazine(java.lang.Thread currentThread)
private boolean
offerToQueue(AdaptivePoolingAllocator.Chunk buffer)
(package private) static int
sizeBucket(int size)
private boolean
tryExpandMagazines(int currentLength)
long
usedMemory()
-
-
-
Field Detail
-
EXPANSION_ATTEMPTS
private static final int EXPANSION_ATTEMPTS
- See Also:
- Constant Field Values
-
INITIAL_MAGAZINES
private static final int INITIAL_MAGAZINES
- See Also:
- Constant Field Values
-
RETIRE_CAPACITY
private static final int RETIRE_CAPACITY
- See Also:
- Constant Field Values
-
MIN_CHUNK_SIZE
private static final int MIN_CHUNK_SIZE
- See Also:
- Constant Field Values
-
MAX_STRIPES
private static final int MAX_STRIPES
-
BUFS_PER_CHUNK
private static final int BUFS_PER_CHUNK
- See Also:
- Constant Field Values
-
MAX_CHUNK_SIZE
private static final int MAX_CHUNK_SIZE
The maximum size of a pooled chunk, in bytes. Allocations bigger than this will never be pooled.This number is 10 MiB, and is derived from the limitations of internal histograms.
- See Also:
- Constant Field Values
-
CENTRAL_QUEUE_CAPACITY
private static final int CENTRAL_QUEUE_CAPACITY
The capacity if the central queue that allow chunks to be shared across magazines. The default size isNettyRuntime.availableProcessors()
, and the maximum number of magazines is twice this.This means the maximum amount of memory that we can have allocated-but-not-in-use is 5 *
NettyRuntime.availableProcessors()
*MAX_CHUNK_SIZE
bytes.
-
MAGAZINE_BUFFER_QUEUE_CAPACITY
private static final int MAGAZINE_BUFFER_QUEUE_CAPACITY
The capacity if the magazine local buffer queue. This queue just pools the outer ByteBuf instance and not the actual memory and so helps to reduce GC pressure.
-
NO_MAGAZINE
private static final java.lang.Object NO_MAGAZINE
-
chunkAllocator
private final AdaptivePoolingAllocator.ChunkAllocator chunkAllocator
-
centralQueue
private final java.util.Queue<AdaptivePoolingAllocator.Chunk> centralQueue
-
magazineExpandLock
private final java.util.concurrent.locks.StampedLock magazineExpandLock
-
magazines
private volatile AdaptivePoolingAllocator.Magazine[] magazines
-
threadLocalMagazine
private final FastThreadLocal<java.lang.Object> threadLocalMagazine
-
liveCachedMagazines
private final java.util.Set<AdaptivePoolingAllocator.Magazine> liveCachedMagazines
-
freed
private volatile boolean freed
-
-
Constructor Detail
-
AdaptivePoolingAllocator
AdaptivePoolingAllocator(AdaptivePoolingAllocator.ChunkAllocator chunkAllocator, AdaptivePoolingAllocator.MagazineCaching magazineCaching)
-
-
Method Detail
-
createSharedChunkQueue
private static java.util.Queue<AdaptivePoolingAllocator.Chunk> createSharedChunkQueue()
Create a thread-safe multi-producer, multi-consumer queue to hold chunks that spill over from the internal Magazines.Each Magazine can only hold two chunks at any one time: the chunk it currently allocates from, and the next-in-line chunk which will be used for allocation once the current one has been used up. This queue will be used by magazines to share any excess chunks they allocate, so that they don't need to allocate new chunks when their current and next-in-line chunks have both been used up.
The simplest implementation of this method is to return a new
ConcurrentLinkedQueue
. However, theCLQ
is unbounded, and this means there's no limit to how many chunks can be cached in this queue.Each chunk in this queue can be up to
MAX_CHUNK_SIZE
in size, so it is recommended to use a bounded queue to limit the maximum memory usage.The default implementation will create a bounded queue with a capacity of
CENTRAL_QUEUE_CAPACITY
.- Returns:
- A new multi-producer, multi-consumer queue.
-
allocate
public ByteBuf allocate(int size, int maxCapacity)
- Specified by:
allocate
in interfaceAdaptiveByteBufAllocator.AdaptiveAllocatorApi
-
allocate
private AdaptivePoolingAllocator.AdaptiveByteBuf allocate(int size, int maxCapacity, java.lang.Thread currentThread, AdaptivePoolingAllocator.AdaptiveByteBuf buf)
-
allocateFallback
private AdaptivePoolingAllocator.AdaptiveByteBuf allocateFallback(int size, int maxCapacity, java.lang.Thread currentThread, AdaptivePoolingAllocator.AdaptiveByteBuf buf)
-
getFallbackMagazine
private AdaptivePoolingAllocator.Magazine getFallbackMagazine(java.lang.Thread currentThread)
-
allocate
void allocate(int size, int maxCapacity, AdaptivePoolingAllocator.AdaptiveByteBuf into)
Allocate into the given buffer. Used byAdaptivePoolingAllocator.AdaptiveByteBuf.capacity(int)
.
-
usedMemory
public long usedMemory()
- Specified by:
usedMemory
in interfaceAdaptiveByteBufAllocator.AdaptiveAllocatorApi
-
tryExpandMagazines
private boolean tryExpandMagazines(int currentLength)
-
offerToQueue
private boolean offerToQueue(AdaptivePoolingAllocator.Chunk buffer)
-
finalize
protected void finalize() throws java.lang.Throwable
- Overrides:
finalize
in classjava.lang.Object
- Throws:
java.lang.Throwable
-
free
private void free()
-
sizeBucket
static int sizeBucket(int size)
-
-