Package io.grpc.rls

Class AdaptiveThrottler

java.lang.Object
io.grpc.rls.AdaptiveThrottler
All Implemented Interfaces:
Throttler

final class AdaptiveThrottler extends Object implements Throttler
Implementation of Throttler that keeps track of recent history (the duration of which is specified to the constructor) and throttles requests at the client side based on the number of requests that the backend has accepted and the total number of requests generated. A given request will be throttled with a probability
   throttleProbability = (requests - ratio_for_accepts * accepts) / (requests + requests_padding)
 
where requests is the total number of requests, accepts is the total number of requests that the backend has accepted and ratio_for_accepts is just a constant multiplier passed to the constructor (see the description of ratio_for_accepts for more information).
  • Field Details

    • DEFAULT_HISTORY_SECONDS

      private static final int DEFAULT_HISTORY_SECONDS
      See Also:
    • DEFAULT_REQUEST_PADDING

      private static final int DEFAULT_REQUEST_PADDING
      See Also:
    • DEFAULT_RATIO_FOR_ACCEPT

      private static final float DEFAULT_RATIO_FOR_ACCEPT
      See Also:
    • historySeconds

      private final int historySeconds
      The duration of history of calls used by Adaptive Throttler.
    • requestsPadding

      private final int requestsPadding
      A magic number to tune the aggressiveness of the throttling. High numbers throttle less. The default is 8.
    • ratioForAccepts

      private final float ratioForAccepts
      The ratio by which the Adaptive Throttler will attempt to send requests above what the server is currently accepting.
    • ticker

      private final com.google.common.base.Ticker ticker
    • requestStat

      The number of requests attempted by the client during the Adaptive Throttler instance's history of calls. This includes requests throttled at the client. The history period defaults to 30 seconds.
    • throttledStat

      Counter for the total number of requests that were throttled by either the client (this class) or the backend in recent history.
  • Constructor Details

  • Method Details

    • shouldThrottle

      public boolean shouldThrottle()
      Description copied from interface: Throttler
      Checks if a given request should be throttled by the client. This should be called for every request before allowing it to hit the network. If the returned value is true, the request should be aborted immediately (as if it had been throttled by the server).

      This updates internal state and should be called exactly once for each request.

      Specified by:
      shouldThrottle in interface Throttler
    • shouldThrottle

      boolean shouldThrottle(float random)
    • getThrottleProbability

      float getThrottleProbability(long nowNanos)
      Calculates throttleProbability.
       throttleProbability = (requests - ratio_for_accepts * accepts) / (requests + requests_padding)
       
    • registerBackendResponse

      public void registerBackendResponse(boolean throttled)
      Description copied from interface: Throttler
      Registers a response received from the backend for a request allowed by shouldThrottle. This should be called for every response received from the backend (i.e., once for each request for which ShouldThrottle returned false). This updates the internal statistics used by shouldThrottle.
      Specified by:
      registerBackendResponse in interface Throttler
      Parameters:
      throttled - specifies whether the request was throttled by the backend.
    • randomFloat

      private static float randomFloat()
    • toString

      public String toString()
      Overrides:
      toString in class Object
    • builder

      public static AdaptiveThrottler.Builder builder()