Class MappedFrontCodedStringBigList
- java.lang.Object
-
- java.util.AbstractCollection<K>
-
- it.unimi.dsi.fastutil.objects.AbstractObjectCollection<K>
-
- it.unimi.dsi.fastutil.objects.AbstractObjectBigList<MutableString>
-
- it.unimi.dsi.big.util.MappedFrontCodedStringBigList
-
- All Implemented Interfaces:
it.unimi.dsi.fastutil.BigList<MutableString>
,it.unimi.dsi.fastutil.objects.ObjectBigList<MutableString>
,it.unimi.dsi.fastutil.objects.ObjectCollection<MutableString>
,it.unimi.dsi.fastutil.objects.ObjectIterable<MutableString>
,it.unimi.dsi.fastutil.Size64
,it.unimi.dsi.fastutil.Stack<MutableString>
,FlyweightPrototype<MappedFrontCodedStringBigList>
,java.io.Closeable
,java.lang.AutoCloseable
,java.lang.Comparable<it.unimi.dsi.fastutil.BigList<? extends MutableString>>
,java.lang.Iterable<MutableString>
,java.util.Collection<MutableString>
,java.util.RandomAccess
public class MappedFrontCodedStringBigList extends it.unimi.dsi.fastutil.objects.AbstractObjectBigList<MutableString> implements java.util.RandomAccess, java.io.Closeable, FlyweightPrototype<MappedFrontCodedStringBigList>
A memory-mapped version ofFrontCodedStringBigList
.This class is functionally identical to
FrontCodedStringBigList
, but its data is memory-mapped from disk. Only UTF-8 encoding is supported.To use this class, one first invokes the
build(String, int, Iterator)
method to generate a property file containing metadata, and two files containing strings and string pointers, respectively. Then, theload(String)
method (invoked with the same basename) will return an instance of this class accessing strings and pointers by memory mapping.Note that for consistency with other classes in this package this class implements a big list of mutable strings; however, for greater flexibility it also implements a
getString(long)
method and agetArray(long)
method.If you need to build an instance from a (possibly compressed) stream, we suggest to adapt it using
FileLinesByteArrayIterable.iterator()
.- See Also:
FrontCodedStringBigList
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
MappedFrontCodedStringBigList.PropertyKeys
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String
BYTE_ARRAY_EXTENSION
protected it.unimi.dsi.fastutil.bytes.ByteBigList
byteList
The underlying byte array.protected long
n
The number of strings in the list.protected it.unimi.dsi.fastutil.longs.LongBigList
pointers
The pointers to entire arrays in the list.static java.lang.String
POINTERS_EXTENSION
static java.lang.String
PROPERTIES_EXTENSION
protected int
ratio
The ratio of this front-coded list.static long
serialVersionUID
-
Constructor Summary
Constructors Modifier Constructor Description protected
MappedFrontCodedStringBigList(long n, int ratio, java.lang.String byteBigList, java.lang.String pointers)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description static void
build(java.lang.String basename, int ratio, java.util.Iterator<byte[]> arrays)
Builds and stores a new memory-mapped front-coded big string list.void
close()
MappedFrontCodedStringBigList
copy()
Returns a copy of this object, sharing state with this object as much as possible.protected static int
countUTF8Chars(byte[] a)
MutableString
get(long index)
Returns the element at the specified position in this front-coded big list as a mutable string.void
get(long index, MutableString s)
Returns the element at the specified position in this front-coded big list by storing it in a mutable string.byte[]
getArray(long index)
Returns the element at the specified position in this front-coded big list as an UTF-8-coded byte array.java.lang.String
getString(long index)
Returns the element at the specified position in this front-coded big list as a string.static MappedFrontCodedStringBigList
load(java.lang.String basename)
Maps in memory a front-coded string big list starting from a basename.static void
main(java.lang.String[] arg)
long
size64()
-
Methods inherited from class it.unimi.dsi.fastutil.objects.AbstractObjectBigList
add, add, addAll, addAll, addElements, addElements, clear, compareTo, contains, ensureIndex, ensureRestrictedIndex, equals, forEach, getElements, hashCode, indexOf, iterator, lastIndexOf, listIterator, listIterator, peek, pop, push, remove, removeElements, set, setElements, size, size, subList, top, toString
-
Methods inherited from class java.util.AbstractCollection
containsAll, isEmpty, remove, removeAll, retainAll, toArray, toArray
-
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface java.util.Collection
containsAll, isEmpty, parallelStream, remove, removeAll, removeIf, retainAll, stream, toArray, toArray, toArray
-
-
-
-
Field Detail
-
serialVersionUID
public static final long serialVersionUID
- See Also:
- Constant Field Values
-
PROPERTIES_EXTENSION
public static final java.lang.String PROPERTIES_EXTENSION
- See Also:
- Constant Field Values
-
BYTE_ARRAY_EXTENSION
public static final java.lang.String BYTE_ARRAY_EXTENSION
- See Also:
- Constant Field Values
-
POINTERS_EXTENSION
public static final java.lang.String POINTERS_EXTENSION
- See Also:
- Constant Field Values
-
n
protected final long n
The number of strings in the list.
-
ratio
protected final int ratio
The ratio of this front-coded list.
-
byteList
protected it.unimi.dsi.fastutil.bytes.ByteBigList byteList
The underlying byte array.
-
pointers
protected it.unimi.dsi.fastutil.longs.LongBigList pointers
The pointers to entire arrays in the list.
-
-
Method Detail
-
copy
public MappedFrontCodedStringBigList copy()
Description copied from interface:FlyweightPrototype
Returns a copy of this object, sharing state with this object as much as possible.- Specified by:
copy
in interfaceFlyweightPrototype<MappedFrontCodedStringBigList>
- Returns:
- a copy of this object, sharing state with this object as much as possible.
-
build
public static void build(java.lang.String basename, int ratio, java.util.Iterator<byte[]> arrays) throws java.io.IOException, org.apache.commons.configuration2.ex.ConfigurationException
Builds and stores a new memory-mapped front-coded big string list.Given a basename, three file with extensions
PROPERTIES_EXTENSION
,BYTE_ARRAY_EXTENSION
andPOINTERS_EXTENSION
will be generated.After building a list, you can load it using the same basename.
- Parameters:
basename
- the basename of the list.ratio
- the ratio.arrays
- an iterator over byte arrays containing UTF-8 encoded-strings.- Throws:
java.io.IOException
org.apache.commons.configuration2.ex.ConfigurationException
-
load
public static MappedFrontCodedStringBigList load(java.lang.String basename) throws org.apache.commons.configuration2.ex.ConfigurationException, java.io.IOException
Maps in memory a front-coded string big list starting from a basename.- Parameters:
basename
- the basename of a memory-mapped front-coded string big list.- Returns:
- a memory-mapped front-coded string big list.
- Throws:
org.apache.commons.configuration2.ex.ConfigurationException
java.io.IOException
-
get
public MutableString get(long index)
Returns the element at the specified position in this front-coded big list as a mutable string.- Specified by:
get
in interfaceit.unimi.dsi.fastutil.BigList<MutableString>
- Parameters:
index
- an index in the list.- Returns:
- a
MutableString
that will contain the string at the specified position. The string may be freely modified.
-
get
public void get(long index, MutableString s)
Returns the element at the specified position in this front-coded big list by storing it in a mutable string.- Parameters:
index
- an index in the list.s
- a mutable string that will contain the string at the specified position.
-
getString
public java.lang.String getString(long index)
Returns the element at the specified position in this front-coded big list as a string.- Parameters:
index
- an index in the list.- Returns:
- a
String
that will contain the string at the specified position.
-
getArray
public byte[] getArray(long index)
Returns the element at the specified position in this front-coded big list as an UTF-8-coded byte array.- Parameters:
index
- an index in the list.- Returns:
- a byte array representing in UTF-8 encoding the string at the specified position.
-
countUTF8Chars
protected static int countUTF8Chars(byte[] a)
-
size64
public long size64()
- Specified by:
size64
in interfaceit.unimi.dsi.fastutil.Size64
-
close
public void close() throws java.io.IOException
- Specified by:
close
in interfacejava.lang.AutoCloseable
- Specified by:
close
in interfacejava.io.Closeable
- Throws:
java.io.IOException
-
main
public static void main(java.lang.String[] arg) throws java.io.IOException, com.martiansoftware.jsap.JSAPException, org.apache.commons.configuration2.ex.ConfigurationException, java.lang.ClassNotFoundException, java.lang.IllegalArgumentException, java.lang.SecurityException
- Throws:
java.io.IOException
com.martiansoftware.jsap.JSAPException
org.apache.commons.configuration2.ex.ConfigurationException
java.lang.ClassNotFoundException
java.lang.IllegalArgumentException
java.lang.SecurityException
-
-