Class SegmentedFile

  • All Implemented Interfaces:
    java.lang.AutoCloseable

    public class SegmentedFile
    extends java.lang.Object
    implements java.lang.AutoCloseable
    Divides a large file in segments and then reads those (in parallel) using memory mapped files (memory mapped file segments). Basic usage:
    1. Call newBuilder(File) to get a SegmentedFile.Builder or of(File) to get a SegmentedFile directly with the default settings.
    2. Then for each of the segments obtained from segments(): Call newTextLineReader(Segment) and using that read all the lines.
    Multiple threads can be used to read the segments in parallel. Each thread just works on one segment at a time, and there's a new TextLineReader for each segment.
    • Field Detail

      • myFileChannel

        private final java.nio.channels.FileChannel myFileChannel
      • myRandomAccessFile

        private final java.io.RandomAccessFile myRandomAccessFile
    • Constructor Detail

    • Method Detail

      • close

        public void close()
        Specified by:
        close in interface java.lang.AutoCloseable
      • newTextLineReader

        public TextLineReader newTextLineReader​(SegmentedFile.Segment segment)
        Call this once for each file segment, and use the returned TextLineReader to read the file segment. The TextLineReader is not thread safe, and should only be used by a single thread.

        The segment is a range of bytes in the file. The TextLineReader will read the bytes in the range [offset, offset + size). The segments are obtained from segments(). The segments are sorted in ascending order by offset.