Package org.xerial.snappy
Class SnappyOutputStream
java.lang.Object
java.io.OutputStream
org.xerial.snappy.SnappyOutputStream
- All Implemented Interfaces:
Closeable
,Flushable
,AutoCloseable
- Direct Known Subclasses:
SnappyHadoopCompatibleOutputStream
This class implements a stream filter for writing compressed data using
Snappy.
The input data is blocked into 32kb size (in default), and each block is
compressed and then passed to the given OutputStream
.
- snappy codec header defined in
SnappyCodec
(8 bytes) - codec version (4 bytes integer, at least 1) and compatible version (4 bytes integer)
- compressed block 1 : a pair of (compressed data size [4 byte integer. Big-endian], compressed data...)
- compressed block 2
- ...
SnappyOutputStream
cannot be
uncompressed by Snappy.uncompress(byte[])
since the output formats of
Snappy.compress(byte[])
and SnappyOutputStream
are different.
Use SnappyInputStream
for uncompress the data generated by
SnappyOutputStream
.-
Field Summary
FieldsModifier and TypeFieldDescriptionprivate final int
private boolean
(package private) static final int
private boolean
protected byte[]
private final BufferAllocator
private int
static final int
(package private) static final int
protected final OutputStream
protected byte[]
private final BufferAllocator
private int
-
Constructor Summary
ConstructorsConstructorDescriptionSnappyOutputStream
(OutputStream out, int blockSize) SnappyOutputStream
(OutputStream out, int blockSize, BufferAllocatorFactory bufferAllocatorFactory) -
Method Summary
Modifier and TypeMethodDescriptionvoid
close()
close the streamprotected void
protected void
void
flush()
private boolean
hasSufficientOutputBufferFor
(int inputSize) void
Compress the raw byte array data.(package private) static int
readInt
(byte[] buffer, int pos) void
write
(byte[] b, int byteOffset, int byteLength) void
write
(double[] f) Compress the input array datavoid
write
(double[] f, int off, int len) Compress the input double array datavoid
write
(float[] f) Compress the input array datavoid
write
(float[] f, int off, int len) Compress the input float array datavoid
write
(int b) Writes the specified byte to this output stream.void
write
(int[] f) Compress the input array datavoid
write
(int[] f, int off, int len) Compress the input int array datavoid
write
(long[] d) Compress the input array datavoid
write
(long[] d, int off, int len) Compress the input long array datavoid
write
(short[] f) Compress the input array datavoid
write
(short[] f, int off, int len) Compress the input short array dataprotected void
Optionally write a preemble before a block.protected void
protected int
(package private) static void
writeInt
(byte[] dst, int offset, int v) Methods inherited from class java.io.OutputStream
nullOutputStream, write
-
Field Details
-
MAX_BLOCK_SIZE
public static final int MAX_BLOCK_SIZE- See Also:
-
MIN_BLOCK_SIZE
static final int MIN_BLOCK_SIZE- See Also:
-
DEFAULT_BLOCK_SIZE
static final int DEFAULT_BLOCK_SIZE- See Also:
-
out
-
blockSize
private final int blockSize -
inputBufferAllocator
-
outputBufferAllocator
-
inputBuffer
protected byte[] inputBuffer -
outputBuffer
protected byte[] outputBuffer -
inputCursor
private int inputCursor -
outputCursor
private int outputCursor -
headerWritten
private boolean headerWritten -
closed
private boolean closed
-
-
Constructor Details
-
SnappyOutputStream
-
SnappyOutputStream
- Parameters:
out
-blockSize
- byte size of the internal buffer size- Throws:
IllegalArgumentException
- when blockSize is larger than 512 MiB
-
SnappyOutputStream
public SnappyOutputStream(OutputStream out, int blockSize, BufferAllocatorFactory bufferAllocatorFactory)
-
-
Method Details
-
write
- Overrides:
write
in classOutputStream
- Throws:
IOException
-
write
Compress the input long array data- Parameters:
d
- input arrayoff
- offset in the arraylen
- the number of elements in the array to copy- Throws:
IOException
-
write
Compress the input double array data- Parameters:
f
- input arrayoff
- offset in the arraylen
- the number of elements in the array to copy- Throws:
IOException
-
write
Compress the input float array data- Parameters:
f
- input arrayoff
- offset in the arraylen
- the number of elements in the array to copy- Throws:
IOException
-
write
Compress the input int array data- Parameters:
f
- input arrayoff
- offset in the arraylen
- the number of elements in the array to copy- Throws:
IOException
-
write
Compress the input short array data- Parameters:
f
- input arrayoff
- offset in the arraylen
- the number of elements in the array to copy- Throws:
IOException
-
write
Compress the input array data- Parameters:
d
-- Throws:
IOException
-
write
Compress the input array data- Parameters:
f
-- Throws:
IOException
-
write
Compress the input array data- Parameters:
f
-- Throws:
IOException
-
write
Compress the input array data- Parameters:
f
-- Throws:
IOException
-
write
Compress the input array data- Parameters:
f
-- Throws:
IOException
-
hasSufficientOutputBufferFor
private boolean hasSufficientOutputBufferFor(int inputSize) -
rawWrite
Compress the raw byte array data.- Parameters:
array
- array data of any type (e.g., byte[], float[], long[], ...)byteOffset
-byteLength
-- Throws:
IOException
-
write
Writes the specified byte to this output stream. The general contract for write is that one byte is written to the output stream. The byte to be written is the eight low-order bits of the argument b. The 24 high-order bits of b are ignored.- Specified by:
write
in classOutputStream
- Throws:
IOException
-
flush
- Specified by:
flush
in interfaceFlushable
- Overrides:
flush
in classOutputStream
- Throws:
IOException
-
writeInt
static void writeInt(byte[] dst, int offset, int v) -
readInt
static int readInt(byte[] buffer, int pos) -
dumpOutput
- Throws:
IOException
-
compressInput
- Throws:
IOException
-
writeHeader
protected int writeHeader() -
writeBlockPreemble
protected void writeBlockPreemble()Optionally write a preemble before a block. Hadoop requires the actual block data size being written. This base implementation does nothing. Derive implementation can callwriteCurrentDataSize()
. -
writeCurrentDataSize
protected void writeCurrentDataSize() -
close
close the stream- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceCloseable
- Overrides:
close
in classOutputStream
- Throws:
IOException
-