Package io.grpc.rls
Class AdaptiveThrottler
- java.lang.Object
-
- io.grpc.rls.AdaptiveThrottler
-
- All Implemented Interfaces:
Throttler
final class AdaptiveThrottler extends java.lang.Object implements Throttler
Implementation ofThrottler
that keeps track of recent history (the duration of which is specified to the constructor) and throttles requests at the client side based on the number of requests that the backend has accepted and the total number of requests generated. A given request will be throttled with a probabilitythrottleProbability = (requests - ratio_for_accepts * accepts) / (requests + requests_padding)
where requests is the total number of requests, accepts is the total number of requests that the backend has accepted and ratio_for_accepts is just a constant multiplier passed to the constructor (see the description of ratio_for_accepts for more information).
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description (package private) static class
AdaptiveThrottler.Builder
Builder forAdaptiveThrottler
.(package private) static class
AdaptiveThrottler.TimeBasedAccumulator
-
Field Summary
Fields Modifier and Type Field Description private static int
DEFAULT_HISTORY_SECONDS
private static float
DEFAULT_RATIO_FOR_ACCEPT
private static int
DEFAULT_REQUEST_PADDING
private int
historySeconds
The duration of history of calls used by Adaptive Throttler.private float
ratioForAccepts
The ratio by which the Adaptive Throttler will attempt to send requests above what the server is currently accepting.private int
requestsPadding
A magic number to tune the aggressiveness of the throttling.(package private) AdaptiveThrottler.TimeBasedAccumulator
requestStat
The number of requests attempted by the client during the Adaptive Throttler instance's history of calls.(package private) AdaptiveThrottler.TimeBasedAccumulator
throttledStat
Counter for the total number of requests that were throttled by either the client (this class) or the backend in recent history.private com.google.common.base.Ticker
ticker
-
Constructor Summary
Constructors Modifier Constructor Description private
AdaptiveThrottler(AdaptiveThrottler.Builder builder)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description static AdaptiveThrottler.Builder
builder()
(package private) float
getThrottleProbability(long nowNanos)
Calculates throttleProbability.private static float
randomFloat()
void
registerBackendResponse(boolean throttled)
Registers a response received from the backend for a request allowed by shouldThrottle.boolean
shouldThrottle()
Checks if a given request should be throttled by the client.(package private) boolean
shouldThrottle(float random)
java.lang.String
toString()
-
-
-
Field Detail
-
DEFAULT_HISTORY_SECONDS
private static final int DEFAULT_HISTORY_SECONDS
- See Also:
- Constant Field Values
-
DEFAULT_REQUEST_PADDING
private static final int DEFAULT_REQUEST_PADDING
- See Also:
- Constant Field Values
-
DEFAULT_RATIO_FOR_ACCEPT
private static final float DEFAULT_RATIO_FOR_ACCEPT
- See Also:
- Constant Field Values
-
historySeconds
private final int historySeconds
The duration of history of calls used by Adaptive Throttler.
-
requestsPadding
private final int requestsPadding
A magic number to tune the aggressiveness of the throttling. High numbers throttle less. The default is 8.
-
ratioForAccepts
private final float ratioForAccepts
The ratio by which the Adaptive Throttler will attempt to send requests above what the server is currently accepting.
-
ticker
private final com.google.common.base.Ticker ticker
-
requestStat
final AdaptiveThrottler.TimeBasedAccumulator requestStat
The number of requests attempted by the client during the Adaptive Throttler instance's history of calls. This includes requests throttled at the client. The history period defaults to 30 seconds.
-
throttledStat
final AdaptiveThrottler.TimeBasedAccumulator throttledStat
Counter for the total number of requests that were throttled by either the client (this class) or the backend in recent history.
-
-
Constructor Detail
-
AdaptiveThrottler
private AdaptiveThrottler(AdaptiveThrottler.Builder builder)
-
-
Method Detail
-
shouldThrottle
public boolean shouldThrottle()
Description copied from interface:Throttler
Checks if a given request should be throttled by the client. This should be called for every request before allowing it to hit the network. If the returned value is true, the request should be aborted immediately (as if it had been throttled by the server).This updates internal state and should be called exactly once for each request.
- Specified by:
shouldThrottle
in interfaceThrottler
-
shouldThrottle
boolean shouldThrottle(float random)
-
getThrottleProbability
float getThrottleProbability(long nowNanos)
Calculates throttleProbability.throttleProbability = (requests - ratio_for_accepts * accepts) / (requests + requests_padding)
-
registerBackendResponse
public void registerBackendResponse(boolean throttled)
Description copied from interface:Throttler
Registers a response received from the backend for a request allowed by shouldThrottle. This should be called for every response received from the backend (i.e., once for each request for which ShouldThrottle returned false). This updates the internal statistics used by shouldThrottle.- Specified by:
registerBackendResponse
in interfaceThrottler
- Parameters:
throttled
- specifies whether the request was throttled by the backend.
-
randomFloat
private static float randomFloat()
-
toString
public java.lang.String toString()
- Overrides:
toString
in classjava.lang.Object
-
builder
public static AdaptiveThrottler.Builder builder()
-
-