Package org.agrona

Class MarkFile

  • All Implemented Interfaces:
    java.lang.AutoCloseable

    public class MarkFile
    extends java.lang.Object
    implements java.lang.AutoCloseable
    A MarkFile is used to mark the presence of a running component and to track liveness.

    The assumptions are: (1) the version field is an int in size, (2) the timestamp field is a long in size, and (3) the version field comes before the timestamp field.

    • Constructor Summary

      Constructors 
      Constructor Description
      MarkFile​(java.io.File markFile, boolean shouldPreExist, int versionFieldOffset, int timestampFieldOffset, int totalFileLength, long timeoutMs, EpochClock epochClock, java.util.function.IntConsumer versionCheck, java.util.function.Consumer<java.lang.String> logger)
      Create a MarkFile if none present.
      MarkFile​(java.io.File directory, java.lang.String filename, boolean warnIfDirectoryExists, boolean dirDeleteOnStart, int versionFieldOffset, int timestampFieldOffset, int totalFileLength, long timeoutMs, EpochClock epochClock, java.util.function.IntConsumer versionCheck, java.util.function.Consumer<java.lang.String> logger)
      Create a directory and mark file if none present.
      MarkFile​(java.io.File directory, java.lang.String filename, int versionFieldOffset, int timestampFieldOffset, long timeoutMs, EpochClock epochClock, java.util.function.IntConsumer versionCheck, java.util.function.Consumer<java.lang.String> logger)
      Map a pre-existing MarkFile if one present and is active.
      MarkFile​(java.nio.MappedByteBuffer mappedBuffer, int versionFieldOffset, int timestampFieldOffset)
      Manage a MarkFile given a mapped file and offsets of version and timestamp.
      MarkFile​(UnsafeBuffer buffer, int versionFieldOffset, int timestampFieldOffset)
      Manage a MarkFile given a buffer and offsets of version and timestamp.
    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      UnsafeBuffer buffer()
      Returns the underlying UnsafeBuffer.
      void close()
      void deleteDirectory​(boolean ignoreFailures)
      Delete parent directory.
      static void ensureDirectoryExists​(java.io.File directory, java.lang.String filename, boolean warnIfDirectoryExists, boolean dirDeleteOnStart, int versionFieldOffset, int timestampFieldOffset, long timeoutMs, EpochClock epochClock, java.util.function.IntConsumer versionCheck, java.util.function.Consumer<java.lang.String> logger)
      Ensure the directory exists, i.e.
      static void ensureMarkFileLink​(java.io.File serviceDir, java.io.File actualFile, java.lang.String linkFilename)
      Ensure a link file exists if required for the actual mark file.
      static boolean isActive​(java.nio.MappedByteBuffer byteBuffer, EpochClock epochClock, long timeoutMs, int versionFieldOffset, int timestampFieldOffset, java.util.function.IntConsumer versionCheck, java.util.function.Consumer<java.lang.String> logger)
      Check if MarkFile is active, i.e.
      boolean isClosed()
      Checks if MarkFile is closed.
      static java.nio.MappedByteBuffer mapExistingFile​(java.io.File markFile, java.util.function.Consumer<java.lang.String> logger, long offset, long length)
      Map existing MarkFile.
      static java.nio.MappedByteBuffer mapExistingMarkFile​(java.io.File markFile, int versionFieldOffset, int timestampFieldOffset, long timeoutMs, EpochClock epochClock, java.util.function.IntConsumer versionCheck, java.util.function.Consumer<java.lang.String> logger)
      Map existing MarkFile.
      static java.nio.MappedByteBuffer mapNewOrExistingMarkFile​(java.io.File markFile, boolean shouldPreExist, int versionFieldOffset, int timestampFieldOffset, long totalFileLength, long timeoutMs, EpochClock epochClock, java.util.function.IntConsumer versionCheck, java.util.function.Consumer<java.lang.String> logger)
      Map new of existing MarkFile.
      java.nio.MappedByteBuffer mappedByteBuffer()
      Returns the underlying MappedByteBuffer.
      java.io.File markFile()
      Returns MarkFile.
      java.io.File parentDirectory()
      Returns parent directory.
      void signalReady​(int version)
      Perform an ordered put of the version field.
      protected static void sleep​(long durationMs)
      Put thread to sleep for the given duration and restore interrupted status if thread is interrupted while sleeping.
      void timestampOrdered​(long timestamp)
      Set timestamp field using an ordered put.
      long timestampVolatile()
      Perform volatile read of the timestamp field.
      long timestampWeak()
      Perform weak/plain read of the timestamp field.
      private static void validateOffsets​(int versionFieldOffset, int timestampFieldOffset)  
      int versionVolatile()
      Perform volatile read of the version field.
      int versionWeak()
      Perform weak/plain read of the version field.
      static java.nio.MappedByteBuffer waitForFileMapping​(java.util.function.Consumer<java.lang.String> logger, java.io.File markFile, long deadlineMs, EpochClock epochClock)
      Await the creation of the MarkFile.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • versionFieldOffset

        private final int versionFieldOffset
      • timestampFieldOffset

        private final int timestampFieldOffset
      • parentDir

        private final java.io.File parentDir
      • markFile

        private final java.io.File markFile
      • mappedBuffer

        private final java.nio.MappedByteBuffer mappedBuffer
      • isClosed

        private final java.util.concurrent.atomic.AtomicBoolean isClosed
    • Constructor Detail

      • MarkFile

        public MarkFile​(java.io.File directory,
                        java.lang.String filename,
                        boolean warnIfDirectoryExists,
                        boolean dirDeleteOnStart,
                        int versionFieldOffset,
                        int timestampFieldOffset,
                        int totalFileLength,
                        long timeoutMs,
                        EpochClock epochClock,
                        java.util.function.IntConsumer versionCheck,
                        java.util.function.Consumer<java.lang.String> logger)
        Create a directory and mark file if none present. Checking if an active Mark file exists and is active. Old Mark file is deleted and recreated if not active.

        Total length of Mark file will be mapped until close() is called.

        Parameters:
        directory - for the Mark file.
        filename - of the Mark file.
        warnIfDirectoryExists - for logging purposes.
        dirDeleteOnStart - if desired.
        versionFieldOffset - to use for version field access.
        timestampFieldOffset - to use for timestamp field access.
        totalFileLength - to allocate when creating new Mark file.
        timeoutMs - for the activity check (in milliseconds).
        epochClock - to use for time checks.
        versionCheck - to use for existing Mark file and version field.
        logger - to use to signal progress or null.
      • MarkFile

        public MarkFile​(java.io.File markFile,
                        boolean shouldPreExist,
                        int versionFieldOffset,
                        int timestampFieldOffset,
                        int totalFileLength,
                        long timeoutMs,
                        EpochClock epochClock,
                        java.util.function.IntConsumer versionCheck,
                        java.util.function.Consumer<java.lang.String> logger)
        Create a MarkFile if none present. Checking if an active MarkFile exists and is active. Existing MarkFile is used if not active.

        Total length of Mark file will be mapped until close() is called.

        Parameters:
        markFile - to use.
        shouldPreExist - or not.
        versionFieldOffset - to use for version field access.
        timestampFieldOffset - to use for timestamp field access.
        totalFileLength - to allocate when creating new MarkFile.
        timeoutMs - for the activity check (in milliseconds).
        epochClock - to use for time checks.
        versionCheck - to use for existing MarkFile and version field.
        logger - to use to signal progress or null.
      • MarkFile

        public MarkFile​(java.io.File directory,
                        java.lang.String filename,
                        int versionFieldOffset,
                        int timestampFieldOffset,
                        long timeoutMs,
                        EpochClock epochClock,
                        java.util.function.IntConsumer versionCheck,
                        java.util.function.Consumer<java.lang.String> logger)
        Map a pre-existing MarkFile if one present and is active.

        Total length of MarkFile will be mapped until close() is called.

        Parameters:
        directory - for the MarkFile file.
        filename - of the MarkFile file.
        versionFieldOffset - to use for version field access.
        timestampFieldOffset - to use for timestamp field access.
        timeoutMs - for the activity check (in milliseconds) and for how long to wait for file to exist.
        epochClock - to use for time checks.
        versionCheck - to use for existing MarkFile file and version field.
        logger - to use to signal progress or null.
      • MarkFile

        public MarkFile​(java.nio.MappedByteBuffer mappedBuffer,
                        int versionFieldOffset,
                        int timestampFieldOffset)
        Manage a MarkFile given a mapped file and offsets of version and timestamp.

        If mappedBuffer is not null, then it will be unmapped upon close().

        Parameters:
        mappedBuffer - for the MarkFile fields.
        versionFieldOffset - for the version field.
        timestampFieldOffset - for the timestamp field.
      • MarkFile

        public MarkFile​(UnsafeBuffer buffer,
                        int versionFieldOffset,
                        int timestampFieldOffset)
        Manage a MarkFile given a buffer and offsets of version and timestamp.
        Parameters:
        buffer - for the MarkFile fields
        versionFieldOffset - for the version field
        timestampFieldOffset - for the timestamp field
    • Method Detail

      • isClosed

        public boolean isClosed()
        Checks if MarkFile is closed.
        Returns:
        true if MarkFile is closed.
      • close

        public void close()
        Specified by:
        close in interface java.lang.AutoCloseable
      • signalReady

        public void signalReady​(int version)
        Perform an ordered put of the version field.
        Parameters:
        version - to be signaled.
      • versionVolatile

        public int versionVolatile()
        Perform volatile read of the version field.
        Returns:
        value of the version field.
      • versionWeak

        public int versionWeak()
        Perform weak/plain read of the version field.
        Returns:
        value of the version field.
      • timestampOrdered

        public void timestampOrdered​(long timestamp)
        Set timestamp field using an ordered put.
        Parameters:
        timestamp - to be set.
      • timestampVolatile

        public long timestampVolatile()
        Perform volatile read of the timestamp field.
        Returns:
        value of the timestamp field.
      • timestampWeak

        public long timestampWeak()
        Perform weak/plain read of the timestamp field.
        Returns:
        value of the timestamp field.
      • deleteDirectory

        public void deleteDirectory​(boolean ignoreFailures)
        Delete parent directory.
        Parameters:
        ignoreFailures - should the failures be silently ignored.
      • parentDirectory

        public java.io.File parentDirectory()
        Returns parent directory.
        Returns:
        parent directory.
      • markFile

        public java.io.File markFile()
        Returns MarkFile.
        Returns:
        MarkFile.
      • mappedByteBuffer

        public java.nio.MappedByteBuffer mappedByteBuffer()
        Returns the underlying MappedByteBuffer.
        Returns:
        reference to the MappedByteBuffer.
      • ensureDirectoryExists

        public static void ensureDirectoryExists​(java.io.File directory,
                                                 java.lang.String filename,
                                                 boolean warnIfDirectoryExists,
                                                 boolean dirDeleteOnStart,
                                                 int versionFieldOffset,
                                                 int timestampFieldOffset,
                                                 long timeoutMs,
                                                 EpochClock epochClock,
                                                 java.util.function.IntConsumer versionCheck,
                                                 java.util.function.Consumer<java.lang.String> logger)
        Ensure the directory exists, i.e. create if it does not exist yet and re-create if it already exists.
        Parameters:
        directory - to create.
        filename - of the MarkFile.
        warnIfDirectoryExists - should print warning if directory already exists.
        dirDeleteOnStart - should directory be deleted if it already exists. When the flag is set to false the check will be made to see if the MarkFile is active.

        Note: the directory will be deleted anyway even if the flag is false.

        versionFieldOffset - offset of the version field.
        timestampFieldOffset - offset of the timestamp field.
        timeoutMs - timeout in milliseconds.
        epochClock - epoch clock.
        versionCheck - MarkFile version check function.
        logger - to use for reporting warnings.
        Throws:
        java.lang.IllegalStateException - if MarkFile already exists and is active and dirDeleteOnStart=false.
      • waitForFileMapping

        public static java.nio.MappedByteBuffer waitForFileMapping​(java.util.function.Consumer<java.lang.String> logger,
                                                                   java.io.File markFile,
                                                                   long deadlineMs,
                                                                   EpochClock epochClock)
        Await the creation of the MarkFile.
        Parameters:
        logger - to use for warnings.
        markFile - the MarkFile.
        deadlineMs - deadline timeout in milliseconds.
        epochClock - epoch clock.
        Returns:
        MappedByteBuffer for the MarkFile.
        Throws:
        java.lang.IllegalStateException - if deadline timeout is reached.
      • mapExistingMarkFile

        public static java.nio.MappedByteBuffer mapExistingMarkFile​(java.io.File markFile,
                                                                    int versionFieldOffset,
                                                                    int timestampFieldOffset,
                                                                    long timeoutMs,
                                                                    EpochClock epochClock,
                                                                    java.util.function.IntConsumer versionCheck,
                                                                    java.util.function.Consumer<java.lang.String> logger)
        Map existing MarkFile.
        Parameters:
        markFile - the MarkFile.
        versionFieldOffset - offset of the version field.
        timestampFieldOffset - offset of the timestamp field.
        timeoutMs - timeout in milliseconds.
        epochClock - epoch clock.
        versionCheck - version check function.
        logger - for the warnings.
        Returns:
        MappedByteBuffer for the MarkFile.
        Throws:
        java.lang.IllegalStateException - if timeout is reached.
        java.lang.IllegalStateException - if MarkFile has wrong size.
      • mapNewOrExistingMarkFile

        public static java.nio.MappedByteBuffer mapNewOrExistingMarkFile​(java.io.File markFile,
                                                                         boolean shouldPreExist,
                                                                         int versionFieldOffset,
                                                                         int timestampFieldOffset,
                                                                         long totalFileLength,
                                                                         long timeoutMs,
                                                                         EpochClock epochClock,
                                                                         java.util.function.IntConsumer versionCheck,
                                                                         java.util.function.Consumer<java.lang.String> logger)
        Map new of existing MarkFile.
        Parameters:
        markFile - the MarkFile.
        shouldPreExist - should MarkFile already exist.
        versionFieldOffset - offset of the version field.
        timestampFieldOffset - offset of the timestamp field.
        totalFileLength - total file length to be mapped.
        timeoutMs - timeout in milliseconds.
        epochClock - epoch clock.
        versionCheck - version check function.
        logger - for the warnings.
        Returns:
        MappedByteBuffer for the MarkFile.
        Throws:
        java.lang.IllegalStateException - if timeout is reached.
      • mapExistingFile

        public static java.nio.MappedByteBuffer mapExistingFile​(java.io.File markFile,
                                                                java.util.function.Consumer<java.lang.String> logger,
                                                                long offset,
                                                                long length)
        Map existing MarkFile.
        Parameters:
        markFile - the MarkFile.
        logger - for the warnings.
        offset - offset to map at.
        length - to map.
        Returns:
        MappedByteBuffer for the MarkFile.
      • isActive

        public static boolean isActive​(java.nio.MappedByteBuffer byteBuffer,
                                       EpochClock epochClock,
                                       long timeoutMs,
                                       int versionFieldOffset,
                                       int timestampFieldOffset,
                                       java.util.function.IntConsumer versionCheck,
                                       java.util.function.Consumer<java.lang.String> logger)
        Check if MarkFile is active, i.e. still in use.
        Parameters:
        byteBuffer - the MappedByteBuffer.
        epochClock - epoch clock.
        timeoutMs - timeout in milliseconds.
        versionFieldOffset - offset of the version field.
        timestampFieldOffset - offset of the timestamp field.
        versionCheck - version check function.
        logger - for the warnings.
        Returns:
        true if MarkFile is active.
      • ensureMarkFileLink

        public static void ensureMarkFileLink​(java.io.File serviceDir,
                                              java.io.File actualFile,
                                              java.lang.String linkFilename)
        Ensure a link file exists if required for the actual mark file. A link file will contain the pathname of the actual mark file's parent directory. This is useful if the mark file should be stored on a different storage medium to the directory of the service. This will create a file with name of linkFilename in the serviceDir. If actualFile is an immediate child of serviceDir then any file with the name of linkFilename will be deleted from the serviceDir (so that links won't be present if not required).
        Parameters:
        serviceDir - directory where the mark file would normally be stored (e.g. archiveDir, clusterDir).
        actualFile - location of actual mark file, e.g. /dev/shm/service/node0/archive-mark.dat
        linkFilename - short name that should be used for the link file, e.g. archive-mark.lnk
      • sleep

        protected static void sleep​(long durationMs)
        Put thread to sleep for the given duration and restore interrupted status if thread is interrupted while sleeping.
        Parameters:
        durationMs - sleep duration in milliseconds.
      • validateOffsets

        private static void validateOffsets​(int versionFieldOffset,
                                            int timestampFieldOffset)