Class ByteDiskQueue

  • All Implemented Interfaces:
    it.unimi.dsi.fastutil.Size64, java.io.Closeable, java.lang.AutoCloseable

    public class ByteDiskQueue
    extends java.lang.Object
    implements java.io.Closeable, it.unimi.dsi.fastutil.Size64
    A queue of bytes partially stored on disk.

    Instances of this class keep track of a queue of bytes using a circular (and possibly direct) ByteBuffer that contains the head and the tail of the queue. The central part of the queue, if necessary, is dumped on disk. Note that clear() does empty the queue, but it does not remove the dump file on disk: use trim() to trim the file at the smallest possible length. The dump file can be removed only with a call to close(), or when the instance is finalised.

    Since file handles are a precious resource, this class provides a suspend() method that releases the file handle without deleting the dump file. All calls to methods accessing the file handle will reinitialise it lazily (which implies that a certain number of enqueues/dequeues can be performed on a suspended class without actually reopening the dump file).

    You can create a queue using a new or empty file, or using an existing file, in which case the content of the file will become the content of the queue. A freeze() method closes a queue but leaves behind a dump file containing the current content of the queue, so that it can be reopened later.

    Warning: the close() method will delete the queue file from disk. This implies that if a file with the same name has been created in the meanwhile, it will be deleted.

    • Constructor Summary

      Constructors 
      Modifier Constructor Description
      protected ByteDiskQueue​(java.io.File file, int bufferSize, boolean direct, boolean useFileContent)
      Creates a new disk-based byte queue.
    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods 
      Modifier and Type Method Description
      void clear()
      Clears the queue.
      void close()
      Deallocates all disk resources associated with this queue.
      static ByteDiskQueue createFromFile​(java.io.File file, int bufferSize, boolean direct)
      Creates a new disk-based byte queue using the content of an existing file.
      static ByteDiskQueue createNew​(java.io.File file, int bufferSize, boolean direct)
      Creates a new empty disk-based byte queue.
      byte dequeue()
      Dequeues a byte from the queue.
      void dequeue​(byte[] a)
      Dequeues a sequence of bytes from this queue into an array.
      void dequeue​(byte[] a, int offset, int length)
      Dequeues a sequence of bytes from this queue into an array.
      int dequeueInt()
      Dequeues from this queue a vByte-coded natural number.
      void enlargeBuffer​(int newBufferSize)
      Enlarge the size of the buffer of this queue to a given size.
      void enqueue​(byte b)
      Enqueues a byte to this queue.
      void enqueue​(byte[] a)
      Enqueues a byte array to this queue.
      void enqueue​(byte[] a, int offset, int length)
      Enqueues a fragment of byte array to this queue.
      void enqueueInt​(int x)
      Adds a vByte-coded natural number to this queue.
      protected void finalize()  
      void freeze()  
      boolean isEmpty()  
      int size()
      Deprecated.
      long size64()  
      void suspend()
      Suspends this queue, releasing the associated file handles.
      void trim()
      Trims the queue dump file at the current write position.
      • Methods inherited from class java.lang.Object

        clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Constructor Detail

      • ByteDiskQueue

        protected ByteDiskQueue​(java.io.File file,
                                int bufferSize,
                                boolean direct,
                                boolean useFileContent)
                         throws java.io.IOException
        Creates a new disk-based byte queue.
        Parameters:
        file - the file that will be used to dump the central part of the queue on disk.
        bufferSize - the capacity of the circular buffer (will be possibly decreased so to be a power of two).
        direct - whether the circular ByteBuffer used by this queue should be allocated directly.
        useFileContent - whether the queue is new or should use the content of an existing file; in the first case, we check that the file does not exists or has length zero.
        Throws:
        java.io.IOException
    • Method Detail

      • createNew

        public static ByteDiskQueue createNew​(java.io.File file,
                                              int bufferSize,
                                              boolean direct)
                                       throws java.io.IOException
        Creates a new empty disk-based byte queue.
        Parameters:
        file - the file that will be used to dump the central part of the queue on disk (must not exist or have length zero).
        bufferSize - the capacity of the circular buffer (will be possibly decreased so to be a power of two).
        direct - whether the circular ByteBuffer used by this queue should be allocated directly.
        Throws:
        java.io.IOException
      • createFromFile

        public static ByteDiskQueue createFromFile​(java.io.File file,
                                                   int bufferSize,
                                                   boolean direct)
                                            throws java.io.IOException
        Creates a new disk-based byte queue using the content of an existing file. The stream of bytes contained in the provided file will form the initial content of the queue.
        Parameters:
        file - the file that will be used to dump the central part of the queue on disk (must exist).
        bufferSize - the capacity of the circular buffer (will be possibly decreased so to be a power of two).
        direct - whether the circular ByteBuffer used by this queue should be allocated directly.
        Throws:
        java.io.IOException
      • enqueue

        public void enqueue​(byte b)
                     throws java.io.IOException
        Enqueues a byte to this queue.
        Parameters:
        b - the byte to be enqueued.
        Throws:
        java.io.IOException
      • enqueueInt

        public void enqueueInt​(int x)
                        throws java.io.IOException
        Adds a vByte-coded natural number to this queue.
        Parameters:
        x - the natural number to be added to the queue.
        Throws:
        java.io.IOException
      • enqueue

        public void enqueue​(byte[] a,
                            int offset,
                            int length)
                     throws java.io.IOException
        Enqueues a fragment of byte array to this queue.
        Parameters:
        a - a byte array.
        offset - the first byte in a to be enqueued.
        length - the number of bytes to enqueue.
        Throws:
        java.io.IOException
      • enqueue

        public void enqueue​(byte[] a)
                     throws java.io.IOException
        Enqueues a byte array to this queue.
        Parameters:
        a - the array whose bytes have to be enqueued.
        Throws:
        java.io.IOException
      • dequeue

        public byte dequeue()
                     throws java.io.IOException
        Dequeues a byte from the queue.
        Returns:
        the first available byte in this queue.
        Throws:
        java.util.NoSuchElementException - if there are no bytes in this queue.
        java.io.IOException
      • dequeue

        public void dequeue​(byte[] a,
                            int offset,
                            int length)
                     throws java.io.IOException
        Dequeues a sequence of bytes from this queue into an array.
        Parameters:
        a - a byte array.
        offset - the first position in a where dequeued bytes will be written.
        length - the number of bytes to dequeue.
        Throws:
        java.util.NoSuchElementException - if there are not enough bytes in this queue.
        java.io.IOException
      • dequeue

        public void dequeue​(byte[] a)
                     throws java.io.IOException
        Dequeues a sequence of bytes from this queue into an array.
        Parameters:
        a - the array that will contain the dequeued bytes.
        Throws:
        java.util.NoSuchElementException - if there are not enough bytes in this queue.
        java.io.IOException
      • dequeueInt

        public int dequeueInt()
                       throws java.io.IOException
        Dequeues from this queue a vByte-coded natural number.
        Returns:
        the first available natural number in this queue.
        Throws:
        java.io.IOException
      • size

        @Deprecated
        public int size()
        Deprecated.
        Specified by:
        size in interface it.unimi.dsi.fastutil.Size64
      • size64

        public long size64()
        Specified by:
        size64 in interface it.unimi.dsi.fastutil.Size64
      • isEmpty

        public boolean isEmpty()
      • close

        public void close()
                   throws java.io.IOException
        Deallocates all disk resources associated with this queue. After calling this method, calls to any methods that would modify the queue will cause an IllegalStateException.
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Throws:
        java.io.IOException
      • freeze

        public void freeze()
                    throws java.io.IOException
        Throws:
        java.io.IOException
      • clear

        public void clear()
        Clears the queue. Note that we do not modify the dump file (use trim() to that purpose).
      • trim

        public void trim()
                  throws java.io.IOException
        Trims the queue dump file at the current write position.
        Throws:
        java.io.IOException
      • suspend

        public void suspend()
                     throws java.io.IOException
        Suspends this queue, releasing the associated file handles. If the deque is already suspended, does nothing. The queue will lazily resume its operations when necessary.
        Throws:
        java.io.IOException
      • finalize

        protected void finalize()
                         throws java.lang.Throwable
        Overrides:
        finalize in class java.lang.Object
        Throws:
        java.lang.Throwable
      • enlargeBuffer

        public void enlargeBuffer​(int newBufferSize)
        Enlarge the size of the buffer of this queue to a given size.
        Parameters:
        newBufferSize - the required buffer size (will be possibly decreased so to be a power of two). If the new buffer size is smaller than the current size, nothing happens.