Class AsynchronousLogShipper
java.lang.Object
java.lang.Thread
org.apache.derby.impl.store.replication.master.AsynchronousLogShipper
- All Implemented Interfaces:
Runnable
,LogShipper
Does asynchronous shipping of log records from the master to the slave being replicated to. The implementation does not ship log records as soon as they become available in the log buffer (synchronously), instead it does log shipping in the following two-fold scenarios 1) Periodically (i.e.) at regular intervals of time. 2) when a request is sent from the master controller (force flushing of the log buffer). 3) when a notification is received from the log shipper about a log buffer element becoming full and the load on the log buffer so warrants a ship.
-
Nested Class Summary
Nested classes/interfaces inherited from class java.lang.Thread
Thread.Builder, Thread.State, Thread.UncaughtExceptionHandler
-
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final int
The number of millis a call to forceFlush will wait before giving up sending a chunk of log to the slaveprivate ReplicationMessage
Store the log chunk that failed during a previous shipping attempt so that it can be re-shipped to the slave.private long
The highest log instant in failedChunkprivate static final int
Fill information value indicative of a high load in the log buffer.private static final int
Fill information value indicative of a low load in the log buffer.private Object
Used to synchronize forceFlush callsprivate long
The highest log instant shipped so farprivate long
Will store the time at which the last shipping happened.private final ReplicationLogBuffer
Replication log buffer that contains the log records that need to be transmitted to the slave.private MasterController
The master controller that initialized this log shipper.private static final long
If the fill information is less than FI_LOW the log shipper will ship with a MAX ms delay or when a buffer becomes full whichever comes first.private long
Minimum interval (in milliseconds) between log shipping.private static final long
If the fill information (obtained from the log buffer) is less than FI_HIGH but greater than FI_LOW the log shipper will ship with a MIN ms delay.private long
Minimum interval (in milliseconds) between log shipping.private Object
Object used to synchronize on while the log shipper thread is moved into the wait state, or while notifying it.private final ReplicationLogger
private long
Time interval (in milliseconds) at which the log shipping takes place.private boolean
Indicates whether a stop shipping request has been sent.private ReplicationMessageTransmit
Replication message transmitter that is used for the network transmission of the log records retrieved from the log buffer (on the master) to the slave being replicated to.Fields inherited from class java.lang.Thread
MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY
-
Constructor Summary
ConstructorsConstructorDescriptionAsynchronousLogShipper
(ReplicationLogBuffer logBuffer, ReplicationMessageTransmit transmitter, MasterController masterController, ReplicationLogger repLogger) Constructor initializes the log buffer, the replication message transmitter, the shipping interval and the master controller. -
Method Summary
Modifier and TypeMethodDescriptionprivate long
Will be used to calculate the shipping interval based on the fill information obtained from the log buffer.void
Transmits all the log records in the log buffer to the slave.void
flushedInstance
(long latestInstanceFlushedToDisk) updates the information about the latest instance of the log record that has been flushed to the disk.void
Transmits a chunk of log record from the log buffer to the slave, used by the master controller when the log buffer is full and some space needs to be freed for further log records.long
Get the highest log instant shipped so farprivate void
Load relevant system properties: max and min log shipping intervalvoid
run()
Ships log records from the log buffer to the slave being replicated to.private boolean
Retrieves a chunk of log records, if available, from the log buffer and transmits them to the slave.void
Stop shipping log records.void
workToDo()
Used to notify the log shipper that a log buffer element is full.Methods inherited from class java.lang.Thread
activeCount, checkAccess, clone, countStackFrames, currentThread, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, isVirtual, join, join, join, join, ofPlatform, ofVirtual, onSpinWait, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, sleep, start, startVirtualThread, stop, suspend, threadId, toString, yield
-
Field Details
-
logBuffer
Replication log buffer that contains the log records that need to be transmitted to the slave. -
transmitter
Replication message transmitter that is used for the network transmission of the log records retrieved from the log buffer (on the master) to the slave being replicated to. -
shippingInterval
private long shippingIntervalTime interval (in milliseconds) at which the log shipping takes place. -
minShippingInterval
private long minShippingIntervalMinimum interval (in milliseconds) between log shipping. Defaults to MIN, but can be configured using system property derby.replication.minLogShippingInterval- See Also:
-
maxShippingInterval
private long maxShippingIntervalMinimum interval (in milliseconds) between log shipping. Defaults to MAX, but can be configured using system property derby.replication.maxLogShippingInterval- See Also:
-
lastShippingTime
private long lastShippingTimeWill store the time at which the last shipping happened. Will be used to calculate the interval between the log ships upon receiving a notification from the log buffer. -
stopShipping
private volatile boolean stopShippingIndicates whether a stop shipping request has been sent. true - stop shipping log records false - shipping can continue without interruption. -
masterController
The master controller that initialized this log shipper. -
objLSTSync
Object used to synchronize on while the log shipper thread is moved into the wait state, or while notifying it. -
forceFlushSemaphore
Used to synchronize forceFlush calls -
DEFAULT_FORCEFLUSH_TIMEOUT
public static final int DEFAULT_FORCEFLUSH_TIMEOUTThe number of millis a call to forceFlush will wait before giving up sending a chunk of log to the slave- See Also:
-
failedChunk
Store the log chunk that failed during a previous shipping attempt so that it can be re-shipped to the slave. -
failedChunkHighestInstant
private long failedChunkHighestInstantThe highest log instant in failedChunk -
highestShippedInstant
private long highestShippedInstantThe highest log instant shipped so far -
FI_LOW
private static final int FI_LOWFill information value indicative of a low load in the log buffer.- See Also:
-
FI_HIGH
private static final int FI_HIGHFill information value indicative of a high load in the log buffer.- See Also:
-
MIN
private static final long MINIf the fill information (obtained from the log buffer) is less than FI_HIGH but greater than FI_LOW the log shipper will ship with a MIN ms delay. MIN is a value that is only as large as not to affect the performance of the master database significantly.- See Also:
-
MAX
private static final long MAXIf the fill information is less than FI_LOW the log shipper will ship with a MAX ms delay or when a buffer becomes full whichever comes first. The delay however will not be smaller than MIN. max(MAX, DEFAULT_NUMBER_LOG_BUFFERS*MIN) is the maximum delay between a log record is committed at the master until it is replicated to the slave. Hence the default latency should be atleast greater than the maximum latency offered by the choice of MIN, hence MAX > DEFAULT_NUMBER_LOG_BUFFERS*MIN.- See Also:
-
repLogger
-
-
Constructor Details
-
AsynchronousLogShipper
public AsynchronousLogShipper(ReplicationLogBuffer logBuffer, ReplicationMessageTransmit transmitter, MasterController masterController, ReplicationLogger repLogger) Constructor initializes the log buffer, the replication message transmitter, the shipping interval and the master controller.- Parameters:
logBuffer
- the replication log buffer that contains the log record chunks to be transmitted to the slave.transmitter
- the replication message transmitter that is used for network transmission of retrieved log records.masterController
- The master controller that initialized this log shipper.repLogger
- The replication logger that will write messages to the log file (typically derby.log)
-
-
Method Details
-
run
public void run()Ships log records from the log buffer to the slave being replicated to. The log shipping happens between shipping intervals of time, the shipping interval being derived from the fill information (an indicator of load in the log buffer) obtained from the log buffer. The shipping can also be triggered in the following situations, 1) Based on notifications from the log buffer, where the fill information is again used as the basis to decide whether a shipping should happen or not 2) On a forceFlush triggered by the log buffer becoming full and the LogBufferFullException being thrown. -
shipALogChunk
Retrieves a chunk of log records, if available, from the log buffer and transmits them to the slave. Used for both periodic and forced shipping.- Returns:
- true if a chunk of log records was shipped. false if no log records were shipped because log buffer is empty.
- Throws:
IOException
- If an exception occurs while trying to ship the replication message (containing the log records) across the network.StandardException
- If an exception occurs while trying to read log records from the log buffer.
-
flushBuffer
Transmits all the log records in the log buffer to the slave.- Specified by:
flushBuffer
in interfaceLogShipper
- Throws:
IOException
- If an exception occurs while trying to ship the replication message (containing the log records) across the network.StandardException
- If an exception occurs while trying to read log records from the log buffer.
-
forceFlush
Transmits a chunk of log record from the log buffer to the slave, used by the master controller when the log buffer is full and some space needs to be freed for further log records.- Specified by:
forceFlush
in interfaceLogShipper
- Throws:
IOException
- If an exception occurs while trying to ship the replication message (containing the log records) across the network.StandardException
- If an exception occurs while trying to read log records from the log buffer.
-
getHighestShippedInstant
public long getHighestShippedInstant()Get the highest log instant shipped so far- Returns:
- the highest log instant shipped so far
-
flushedInstance
public void flushedInstance(long latestInstanceFlushedToDisk) updates the information about the latest instance of the log record that has been flushed to the disk. Calling this method has no effect in this asynchronous implementation of the log shipper.- Specified by:
flushedInstance
in interfaceLogShipper
- Parameters:
latestInstanceFlushedToDisk
- a long that contains the latest instance of the log record that has been flushed to the disk.
-
stopLogShipment
public void stopLogShipment()Stop shipping log records. If a ship is currently in progress it will not be interrupted, shipping will stop only after the current shipment is done. -
workToDo
public void workToDo()Used to notify the log shipper that a log buffer element is full. This method would basically use the following steps to decide on the action to be taken when a notification from the log shipper is received, a) Get FI from log buffer b) If FI >= FI_HIGH b.1) notify the log shipper thread. c) Else If the time elapsed since last ship is greater than minShippingInterval c.1) notify the log shipper thread.- Specified by:
workToDo
in interfaceLogShipper
-
calculateSIfromFI
private long calculateSIfromFI()Will be used to calculate the shipping interval based on the fill information obtained from the log buffer. This method uses the following steps to arrive at the shipping interval, a) FI >= FI_HIGH return -1 (signifies that the waiting time should be 0) b) FI > FI_LOW and FI < FI_HIGH return minShippingInterval c) FI <= FI_LOW return maxShippingInterval.- Returns:
- the shipping interval based on the fill information.
-
getLogShipperProperties
private void getLogShipperProperties()Load relevant system properties: max and min log shipping interval
-