Class StreamStore
The key of the map is a long (incremented for each stored block). The default initial value is 0. Before storing blocks into the map, the stream store checks if there is already a block with the next key, and if necessary searches the next free entry using a binary search (0 to Long.MAX_VALUE).
Before storing
The format of the binary id is: An empty id represents 0 bytes of data. In-place data is encoded as 0, the size (a variable size int), then the data. A stored block is encoded as 1, the length of the block (a variable size int), then the key (a variable size long). Multiple ids can be concatenated to concatenate the data. If the id is large, it is stored itself, which is encoded as 2, the total length (a variable size long), and the key of the block that contains the id (a variable size long).
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescription(package private) static class
A stream backed by a map. -
Field Summary
FieldsModifier and TypeFieldDescriptionprivate int
private int
private final AtomicReference
<byte[]> private final AtomicLong
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionget
(byte[] id) Open an input stream to read data.private long
Generate a new key.(package private) byte[]
getBlock
(long key) Get the block.getMap()
long
getMaxBlockKey
(byte[] id) Get the key of the biggest block, of -1 for inline data.long
int
long
boolean
isInPlace
(byte[] id) Check whether the id itself contains all the data.long
length
(byte[] id) Calculate the number of data bytes for the given id.protected void
onStore
(int len) This method is called after a block of data is stored.private boolean
put
(ByteArrayOutputStream id, InputStream in, int level) byte[]
put
(InputStream in) Store the stream, and return the id.private ByteArrayOutputStream
private static byte[]
read
(InputStream in, byte[] target) void
remove
(byte[] id) Remove all stored blocks for the given id.void
setMaxBlockSize
(int maxBlockSize) Set the maximum block size.void
setMinBlockSize
(int minBlockSize) Set the minimum block size.void
setNextKey
(long nextKey) static String
toString
(byte[] id) Convert the id to a human readable string.private long
writeBlock
(byte[] data)
-
Field Details
-
map
-
minBlockSize
private int minBlockSize -
maxBlockSize
private int maxBlockSize -
nextKey
-
nextBuffer
-
-
Constructor Details
-
StreamStore
Create a stream store instance.- Parameters:
map
- the map to store blocks of data
-
-
Method Details
-
getMap
-
setNextKey
public void setNextKey(long nextKey) -
getNextKey
public long getNextKey() -
setMinBlockSize
public void setMinBlockSize(int minBlockSize) Set the minimum block size. The default is 256 bytes.- Parameters:
minBlockSize
- the new value
-
getMinBlockSize
public int getMinBlockSize() -
setMaxBlockSize
public void setMaxBlockSize(int maxBlockSize) Set the maximum block size. The default is 256 KB.- Parameters:
maxBlockSize
- the new value
-
getMaxBlockSize
public long getMaxBlockSize() -
put
Store the stream, and return the id. The stream is not closed.- Parameters:
in
- the stream- Returns:
- the id (potentially an empty array)
- Throws:
IOException
- If an I/O error occurs
-
put
- Throws:
IOException
-
read
- Throws:
IOException
-
putIndirectId
- Throws:
IOException
-
writeBlock
private long writeBlock(byte[] data) -
onStore
protected void onStore(int len) This method is called after a block of data is stored. Override this method to persist data if necessary.- Parameters:
len
- the length of the stored block.
-
getAndIncrementNextKey
private long getAndIncrementNextKey()Generate a new key.- Returns:
- the new key
-
getMaxBlockKey
public long getMaxBlockKey(byte[] id) Get the key of the biggest block, of -1 for inline data. This method is used to garbage collect orphaned blocks.- Parameters:
id
- the id- Returns:
- the key, or -1
-
remove
public void remove(byte[] id) Remove all stored blocks for the given id.- Parameters:
id
- the id
-
toString
Convert the id to a human readable string.- Parameters:
id
- the stream id- Returns:
- the string
-
length
public long length(byte[] id) Calculate the number of data bytes for the given id. As the length is encoded in the id, this operation does not cause any reads in the map.- Parameters:
id
- the id- Returns:
- the length
-
isInPlace
public boolean isInPlace(byte[] id) Check whether the id itself contains all the data. This operation does not cause any reads in the map.- Parameters:
id
- the id- Returns:
- if the id contains the data
-
get
Open an input stream to read data.- Parameters:
id
- the id- Returns:
- the stream
-
getBlock
byte[] getBlock(long key) Get the block.- Parameters:
key
- the key- Returns:
- the block
-