Class RAFContainer4
- All Implemented Interfaces:
PrivilegedExceptionAction<Object>
,Cacheable
,TypedFormat
,Lockable
Note that our requests for multiple concurrent IOs may be serialized further down in the IO stack - this is entirely up to the JVM and OS. However, at least in Linux on Sun's 1.4.2_09 JVM we see the desired behavior: The FileChannel.read/write(ByteBuffer buf, long position) calls map to pread/pwrite system calls, which enable efficient IO to the same file descriptor by multiple threads.
This whole class should be merged back into RAFContainer when Derby officially stops supporting Java 1.3.
Significant behavior changes from RAFContainer:
- Multiple concurrent IOs permitted.
- State changes to the container (create, open, close) can now happen while IO is in progress due to the lack of locking. Closing a container while IO is in progress will cause IOExceptions in the thread calling readPage or writePage. If this happens something is probably amiss anyway. The iosInProgress variable is used in an attempt to detect this should it happen while running a debug build.
- See Also:
-
Field Summary
FieldsModifier and TypeFieldDescriptionprivate final Object
private ContainerKey
private boolean
private final Object
private int
For debugging - will be incremented when an IO is started, decremented when it is done.private FileChannel
This channel will be retrieved from RAFContainer's fileData member when fileData is set.private boolean
private int
Fields inherited from class org.apache.derby.impl.store.raw.data.RAFContainer
fileData, needsSync
Fields inherited from class org.apache.derby.impl.store.raw.data.FileContainer
allocCache, canUpdate, CHECKSUM_SIZE, CONTAINER_INFO_SIZE, containerCache, containerInfo, containerVersion, dataFactory, estimatedRowCount, FIRST_ALLOC_PAGE_NUMBER, FIRST_ALLOC_PAGE_OFFSET, firstAllocPageNumber, firstAllocPageOffset, formatIdInteger, initialPages, isDirty, lastLogInstant, minimumRecordSize, pageCache, pageSize, preDirty, SPACE_TRACE, spareSpace
Fields inherited from class org.apache.derby.impl.store.raw.data.BaseContainer
identity, isCommittedDrop, isDropped, isReusableRecordId
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionprivate void
awaitRestoreChannel
(Exception e, boolean stealthMode) Use when seeing an exception during IO and when another thread is presumably doing the recovery.(package private) void
override of RAFContainer#closeContainer(package private) void
createContainer
(ContainerKey newIdentity) override of RAFContainer#createContainerprivate static void
debugTrace
(String msg) private FileChannel
Return the file channel for the current value of thefileData
field.private FileChannel
Return theFileChannel
for the specifiedStorageRandomAccessFile
if it is aRandomAccessFile
.(package private) byte[]
getEmbryonicPage
(StorageRandomAccessFile file, long offset) Read an embryonic page (that is, a section of the first alloc page that is so large that we know all the borrowed space is included in it) from the specified offset in aStorageRandomAccessFile
.private void
handleClosedChannel
(ClosedChannelException e, boolean stealthMode, int retries) This method handles what to do when, during a NIO operation we receive aClosedChannelException
.(package private) boolean
openContainer
(ContainerKey newIdentity) Open a container.private void
readFull
(ByteBuffer dstBuffer, FileChannel srcChannel, long position) Attempts to fill buf completely from start until it's full.protected void
readPage
(long pageNumber, byte[] pageData) Read a page into the supplied array.private void
readPage
(long pageNumber, byte[] pageData, long offset) Read a page into the supplied array.private void
readPage0
(long pageNumber, byte[] pageData, long offset) private boolean
recoverContainerAfterInterrupt
(String whence, boolean stealthMode) Use this when the thread has received a ClosedByInterruptException (or, prior to JDK 1.7 it may also be AsynchronousCloseException - a bug) exception during IO and its interruped flag is also set.private void
reopen()
When the existing channel (ourChannel
) has been closed due to interrupt, we need to reopen the underlying RAF to get a fresh channel so we can resume IO.(package private) void
writeAtOffset
(StorageRandomAccessFile file, byte[] bytes, long offset) Write a sequence of bytes at the given offset in a file.private void
writeFull
(ByteBuffer srcBuffer, FileChannel dstChannel, long position) Attempts to write buf completely from start until end, at the given position in the destination fileChannel.protected void
writePage
(long pageNumber, byte[] pageData, boolean syncPage) Write a page from the supplied array.private void
writePage0
(long pageNumber, byte[] pageData, boolean syncPage) Methods inherited from class org.apache.derby.impl.store.raw.data.RAFContainer
backupContainer, clean, encryptOrDecryptContainer, flushAll, getFileName, getRandomAccessFile, isDirty, preAllocate, privGetFileName, removeContainer, removeFile, reopenContainer, run, truncatePages, updatePageArray
Methods inherited from class org.apache.derby.impl.store.raw.data.FileContainer
bumpContainerVersion, canUpdate, clearIdentity, clearPreallocThreshold, compressContainer, createIdent, createIdentity, deallocatePage, decryptPage, doPreAllocatePages, dropContainer, encryptPage, getAllocPage, getAnyPage, getContainerProperties, getContainerVersion, getContextService, getEmbryonicPage, getEncryptionBuffer, getEstimatedPageCount, getEstimatedRowCount, getFirstHeadPage, getHeadPage, getLastPageNumber, getLatchedPage, getMinimumRecordSize, getNextHeadPage, getPage, getPageForCompress, getPageForInsert, getPageSize, getReusableRecordIdSequenceNumber, getSpaceInfo, getSpareSpace, getTypeFormatId, incrementReusableRecordIdSequenceNumber, initPage, latchPage, letGo, logCreateContainerInfo, newPage, preDirty, prepareForBulkLoad, readHeader, reCreatePageForRedoRecovery, setDirty, setEstimatedRowCount, setIdent, setIdentity, trackUnfilledPage, updateEstimatedRowCount, writeHeader, writeHeader
Methods inherited from class org.apache.derby.impl.store.raw.data.BaseContainer
addPage, compressContainer, fillInIdentity, getAllocPage, getAnyPage, getCommittedDropState, getContainerId, getContainerStatus, getDeallocLock, getDroppedState, getFirstPage, getIdentity, getNextPage, getSegmentId, isReusableRecordId, lockAttributes, lockerAlwaysCompatible, lockEvent, removePage, requestCompatible, setCommittedDropState, setDroppedState, setReusableRecordIdState, truncate, unlockEvent, use
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.derby.iapi.services.cache.Cacheable
getIdentity
-
Field Details
-
ourChannel
This channel will be retrieved from RAFContainer's fileData member when fileData is set. We wrap a couple of RAFContainer's methods to accomplish this. -
channelCleanupMonitor
-
threadsInPageIO
private volatile int threadsInPageIO -
restoreChannelInProgress
private volatile boolean restoreChannelInProgress -
giveUpIO
private boolean giveUpIO -
giveUpIOm
-
iosInProgress
private int iosInProgressFor debugging - will be incremented when an IO is started, decremented when it is done. Should be == 0 when container state is changed. -
currentIdentity
-
-
Constructor Details
-
RAFContainer4
-
-
Method Details
-
getChannel
Return theFileChannel
for the specifiedStorageRandomAccessFile
if it is aRandomAccessFile
. Otherwise, returnnull
.- Parameters:
file
- the file to get the channel for- Returns:
- a
FileChannel
iffile
is an instance ofRandomAccessFile
,null
otherwise
-
getChannel
Return the file channel for the current value of the
fileData
field. IffileData
doesn't support file channels, returnnull
.Callers of this method must synchronize on the container object since two shared fields (
fileData
andourChannel
) are accessed.- Returns:
- a
FileChannel
object, if supported, ornull
-
openContainer
Description copied from class:FileContainer
Open a container.Longer descrption of routine.
Open a container. Open the file that maps to this container, if the file does not exist then we assume the container was never created. If the file exists but we have trouble opening it then we throw some exception.
MT - single thread required - Enforced by cache manager.- Overrides:
openContainer
in classRAFContainer
- Throws:
StandardException
- Standard exception policy.
-
createContainer
override of RAFContainer#createContainer- Overrides:
createContainer
in classRAFContainer
- Throws:
StandardException
- Derby Standard error policy
-
reopen
When the existing channel (ourChannel
) has been closed due to interrupt, we need to reopen the underlying RAF to get a fresh channel so we can resume IO.- Throws:
StandardException
-
closeContainer
void closeContainer()override of RAFContainer#closeContainer- Overrides:
closeContainer
in classRAFContainer
-
readPage
Read a page into the supplied array. override of RAFContainer#readPage
MT - thread safe- Overrides:
readPage
in classRAFContainer
- Throws:
IOException
- exception reading pageStandardException
- Standard Derby error policy
-
readPage
private void readPage(long pageNumber, byte[] pageData, long offset) throws IOException, StandardException Read a page into the supplied array. override of RAFContainer#readPage
MT - thread safe- Parameters:
pageNumber
- the page number to read data from, or -1 (called from getEmbryonicPage)pageData
- the buffer to read data intooffset
- -1 normally (not used since offset is computed from pageNumber), but used if pageNumber == -1 (getEmbryonicPage)- Throws:
IOException
- exception reading pageStandardException
- Standard Derby error policy
-
readPage0
private void readPage0(long pageNumber, byte[] pageData, long offset) throws IOException, StandardException - Throws:
IOException
StandardException
-
writePage
protected void writePage(long pageNumber, byte[] pageData, boolean syncPage) throws IOException, StandardException Write a page from the supplied array. override of RAFContainer#writePage
MT - thread safe- Overrides:
writePage
in classRAFContainer
- Throws:
StandardException
- Standard Derby error policyIOException
- IO error accessing page
-
handleClosedChannel
private void handleClosedChannel(ClosedChannelException e, boolean stealthMode, int retries) throws StandardException This method handles what to do when, during a NIO operation we receive a
ClosedChannelException
. Note the specialization hierarchy:ClosedChannelException
->AsynchronousCloseException
->ClosedByInterruptException
If
e
is a ClosedByInterruptException, we normally start container recovery, i.e. we need to reopen the random access file so we get get a new interruptible channel and continue IO.If
e
is aAsynchronousCloseException
or a plainClosedChannelException
, the behavior depends ofstealthMode
:If
stealthMode == false
, the method will wait for another thread tp finish recovering the IO channel before returning.If
stealthMode == true
, the method throwsInterruptDetectedException
, allowing retry at a higher level in the code. The reason for this is that we sometimes need to release monitors on objects needed by the recovery thread.- Parameters:
e
- Should be an instance ofClosedChannelException
.stealthMode
- Iftrue
, do retry at a higher levelretries
- Give up waiting for another thread to reopen the channel whenretries
reaches 0. Only applicable ifstealthMode == false
.- Throws:
InterruptDetectedException
- if retry at higher level is requiredstealthMode == true
.StandardException
- standard error policy, incl. when we give up waiting for another thread to reopen channel
-
awaitRestoreChannel
Use when seeing an exception during IO and when another thread is presumably doing the recovery. IfstealthMode == false
, wait for another thread to recover the container after an interrupt. IfstealthMode == true
, throw internal exceptionInterruptDetectedException
to do retry from higher in the stack. IfstealthMode == false
, maximum wait time for the container to become available again is determined by the productInterruptStatus.MAX_INTERRUPT_RETRIES * InterruptStatus.INTERRUPT_RETRY_SLEEP
. There is a chance this thread will not see any recovery occuring (yet), in which case it waits for a bit and just returns, so the caller must retry IO until success. If for some reason the recovering thread has given up on resurrecting the container, cf#giveUpIO
, the method throwsFILE_IO_INTERRUPTED
.- Parameters:
e
- the exception we saw during IOstealthMode
- true if the thread doing IO in stealth mode- Throws:
StandardException
-InterruptDetectedException
and normal error policy
-
recoverContainerAfterInterrupt
private boolean recoverContainerAfterInterrupt(String whence, boolean stealthMode) throws StandardException Use this when the thread has received a ClosedByInterruptException (or, prior to JDK 1.7 it may also be AsynchronousCloseException - a bug) exception during IO and its interruped flag is also set. This makes this thread a likely candicate to do container recovery, unless another thread started it already, cf. return value.- Parameters:
whence
- caller site (debug info)stealthMode
- don't update threadsInPageIO if true- Returns:
- true if we did recovery, false if we saw someone else do it and abstained
- Throws:
StandardException
-
writePage0
private void writePage0(long pageNumber, byte[] pageData, boolean syncPage) throws IOException, StandardException - Throws:
IOException
StandardException
-
writeAtOffset
void writeAtOffset(StorageRandomAccessFile file, byte[] bytes, long offset) throws IOException, StandardException Write a sequence of bytes at the given offset in a file. This method operates in stealth mode, see doc forhandleClosedChannel
. This presumes that IO retry happens at a higher level, i.e. the caller(s) must be prepared to handleInterruptDetectedException
. This method overrides FileContainer#writeAtOffset.- Overrides:
writeAtOffset
in classFileContainer
- Parameters:
file
- the file to write tobytes
- the bytes to writeoffset
- the offset to start writing at- Throws:
IOException
- if an I/O error occurs while writingStandardException
- Derby Standard error policy
-
getEmbryonicPage
byte[] getEmbryonicPage(StorageRandomAccessFile file, long offset) throws IOException, StandardException Read an embryonic page (that is, a section of the first alloc page that is so large that we know all the borrowed space is included in it) from the specified offset in aStorageRandomAccessFile
. override of FileContainer#getEmbryonicPage- Overrides:
getEmbryonicPage
in classFileContainer
- Parameters:
file
- the file to read fromoffset
- where to start reading (normallyFileContainer.FIRST_ALLOC_PAGE_OFFSET
)- Returns:
- a byte array containing the embryonic page
- Throws:
IOException
- if an I/O error occurs while readingStandardException
- if thread is interrupted.
-
readFull
private void readFull(ByteBuffer dstBuffer, FileChannel srcChannel, long position) throws IOException, StandardException Attempts to fill buf completely from start until it's full. FileChannel has no readFull() method, so we roll our own.- Parameters:
dstBuffer
- buffer to read intosrcChannel
- channel to read fromposition
- file position from where to read- Throws:
IOException
- if an I/O error occurs while readingStandardException
- If thread is interrupted.
-
writeFull
private void writeFull(ByteBuffer srcBuffer, FileChannel dstChannel, long position) throws IOException Attempts to write buf completely from start until end, at the given position in the destination fileChannel. FileChannel has no writeFull() method, so we roll our own.- Parameters:
srcBuffer
- buffer to writedstChannel
- channel to write toposition
- file position to start writing at- Throws:
IOException
- if an I/O error occurs while writingStandardException
- If thread is interrupted.
-
debugTrace
-