Class AsynchronousLogShipper

java.lang.Object
java.lang.Thread
org.apache.derby.impl.store.replication.master.AsynchronousLogShipper
All Implemented Interfaces:
Runnable, LogShipper

public class AsynchronousLogShipper extends Thread implements LogShipper

Does asynchronous shipping of log records from the master to the slave being replicated to. The implementation does not ship log records as soon as they become available in the log buffer (synchronously), instead it does log shipping in the following two-fold scenarios 1) Periodically (i.e.) at regular intervals of time. 2) when a request is sent from the master controller (force flushing of the log buffer). 3) when a notification is received from the log shipper about a log buffer element becoming full and the load on the log buffer so warrants a ship.

  • Field Details

    • logBuffer

      private final ReplicationLogBuffer logBuffer
      Replication log buffer that contains the log records that need to be transmitted to the slave.
    • transmitter

      private ReplicationMessageTransmit transmitter
      Replication message transmitter that is used for the network transmission of the log records retrieved from the log buffer (on the master) to the slave being replicated to.
    • shippingInterval

      private long shippingInterval
      Time interval (in milliseconds) at which the log shipping takes place.
    • minShippingInterval

      private long minShippingInterval
      Minimum interval (in milliseconds) between log shipping. Defaults to MIN, but can be configured using system property derby.replication.minLogShippingInterval
      See Also:
    • maxShippingInterval

      private long maxShippingInterval
      Minimum interval (in milliseconds) between log shipping. Defaults to MAX, but can be configured using system property derby.replication.maxLogShippingInterval
      See Also:
    • lastShippingTime

      private long lastShippingTime
      Will store the time at which the last shipping happened. Will be used to calculate the interval between the log ships upon receiving a notification from the log buffer.
    • stopShipping

      private volatile boolean stopShipping
      Indicates whether a stop shipping request has been sent. true - stop shipping log records false - shipping can continue without interruption.
    • masterController

      private MasterController masterController
      The master controller that initialized this log shipper.
    • objLSTSync

      private Object objLSTSync
      Object used to synchronize on while the log shipper thread is moved into the wait state, or while notifying it.
    • forceFlushSemaphore

      private Object forceFlushSemaphore
      Used to synchronize forceFlush calls
    • DEFAULT_FORCEFLUSH_TIMEOUT

      public static final int DEFAULT_FORCEFLUSH_TIMEOUT
      The number of millis a call to forceFlush will wait before giving up sending a chunk of log to the slave
      See Also:
    • failedChunk

      private ReplicationMessage failedChunk
      Store the log chunk that failed during a previous shipping attempt so that it can be re-shipped to the slave.
    • failedChunkHighestInstant

      private long failedChunkHighestInstant
      The highest log instant in failedChunk
    • highestShippedInstant

      private long highestShippedInstant
      The highest log instant shipped so far
    • FI_LOW

      private static final int FI_LOW
      Fill information value indicative of a low load in the log buffer.
      See Also:
    • FI_HIGH

      private static final int FI_HIGH
      Fill information value indicative of a high load in the log buffer.
      See Also:
    • MIN

      private static final long MIN
      If the fill information (obtained from the log buffer) is less than FI_HIGH but greater than FI_LOW the log shipper will ship with a MIN ms delay. MIN is a value that is only as large as not to affect the performance of the master database significantly.
      See Also:
    • MAX

      private static final long MAX
      If the fill information is less than FI_LOW the log shipper will ship with a MAX ms delay or when a buffer becomes full whichever comes first. The delay however will not be smaller than MIN. max(MAX, DEFAULT_NUMBER_LOG_BUFFERS*MIN) is the maximum delay between a log record is committed at the master until it is replicated to the slave. Hence the default latency should be atleast greater than the maximum latency offered by the choice of MIN, hence MAX > DEFAULT_NUMBER_LOG_BUFFERS*MIN.
      See Also:
    • repLogger

      private final ReplicationLogger repLogger
  • Constructor Details

    • AsynchronousLogShipper

      public AsynchronousLogShipper(ReplicationLogBuffer logBuffer, ReplicationMessageTransmit transmitter, MasterController masterController, ReplicationLogger repLogger)
      Constructor initializes the log buffer, the replication message transmitter, the shipping interval and the master controller.
      Parameters:
      logBuffer - the replication log buffer that contains the log record chunks to be transmitted to the slave.
      transmitter - the replication message transmitter that is used for network transmission of retrieved log records.
      masterController - The master controller that initialized this log shipper.
      repLogger - The replication logger that will write messages to the log file (typically derby.log)
  • Method Details

    • run

      public void run()
      Ships log records from the log buffer to the slave being replicated to. The log shipping happens between shipping intervals of time, the shipping interval being derived from the fill information (an indicator of load in the log buffer) obtained from the log buffer. The shipping can also be triggered in the following situations, 1) Based on notifications from the log buffer, where the fill information is again used as the basis to decide whether a shipping should happen or not 2) On a forceFlush triggered by the log buffer becoming full and the LogBufferFullException being thrown.
      Specified by:
      run in interface Runnable
      Overrides:
      run in class Thread
    • shipALogChunk

      private boolean shipALogChunk() throws IOException, StandardException
      Retrieves a chunk of log records, if available, from the log buffer and transmits them to the slave. Used for both periodic and forced shipping.
      Returns:
      true if a chunk of log records was shipped. false if no log records were shipped because log buffer is empty.
      Throws:
      IOException - If an exception occurs while trying to ship the replication message (containing the log records) across the network.
      StandardException - If an exception occurs while trying to read log records from the log buffer.
    • flushBuffer

      public void flushBuffer() throws IOException, StandardException
      Transmits all the log records in the log buffer to the slave.
      Specified by:
      flushBuffer in interface LogShipper
      Throws:
      IOException - If an exception occurs while trying to ship the replication message (containing the log records) across the network.
      StandardException - If an exception occurs while trying to read log records from the log buffer.
    • forceFlush

      public void forceFlush() throws IOException, StandardException
      Transmits a chunk of log record from the log buffer to the slave, used by the master controller when the log buffer is full and some space needs to be freed for further log records.
      Specified by:
      forceFlush in interface LogShipper
      Throws:
      IOException - If an exception occurs while trying to ship the replication message (containing the log records) across the network.
      StandardException - If an exception occurs while trying to read log records from the log buffer.
    • getHighestShippedInstant

      public long getHighestShippedInstant()
      Get the highest log instant shipped so far
      Returns:
      the highest log instant shipped so far
    • flushedInstance

      public void flushedInstance(long latestInstanceFlushedToDisk)
      updates the information about the latest instance of the log record that has been flushed to the disk. Calling this method has no effect in this asynchronous implementation of the log shipper.
      Specified by:
      flushedInstance in interface LogShipper
      Parameters:
      latestInstanceFlushedToDisk - a long that contains the latest instance of the log record that has been flushed to the disk.
    • stopLogShipment

      public void stopLogShipment()
      Stop shipping log records. If a ship is currently in progress it will not be interrupted, shipping will stop only after the current shipment is done.
    • workToDo

      public void workToDo()
      Used to notify the log shipper that a log buffer element is full. This method would basically use the following steps to decide on the action to be taken when a notification from the log shipper is received, a) Get FI from log buffer b) If FI >= FI_HIGH b.1) notify the log shipper thread. c) Else If the time elapsed since last ship is greater than minShippingInterval c.1) notify the log shipper thread.
      Specified by:
      workToDo in interface LogShipper
    • calculateSIfromFI

      private long calculateSIfromFI()
      Will be used to calculate the shipping interval based on the fill information obtained from the log buffer. This method uses the following steps to arrive at the shipping interval, a) FI >= FI_HIGH return -1 (signifies that the waiting time should be 0) b) FI > FI_LOW and FI < FI_HIGH return minShippingInterval c) FI <= FI_LOW return maxShippingInterval.
      Returns:
      the shipping interval based on the fill information.
    • getLogShipperProperties

      private void getLogShipperProperties()
      Load relevant system properties: max and min log shipping interval