Package edu.berkeley.nlp.lm.io
Class ArpaLmReader<W>
- java.lang.Object
-
- edu.berkeley.nlp.lm.io.ArpaLmReader<W>
-
- All Implemented Interfaces:
LmReader<ProbBackoffPair,ArpaLmReaderCallback<ProbBackoffPair>>
public class ArpaLmReader<W> extends java.lang.Object implements LmReader<ProbBackoffPair,ArpaLmReaderCallback<ProbBackoffPair>>
A parser for ARPA LM files.- Author:
- Alex Bouchard-Cote, Adam Pauls
-
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String
END_SYMBOL
static java.lang.String
START_SYMBOL
static java.lang.String
UNK_SYMBOL
-
Constructor Summary
Constructors Constructor Description ArpaLmReader(java.lang.String file, WordIndexer<W> wordIndexer, int maxNgramOrder)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
parse(ArpaLmReaderCallback<ProbBackoffPair> callback)
Parse the ARPA file and populate the relevant fields of the enclosing ICSILanguageModelprotected java.util.List<java.lang.Long>
parseHeader()
protected void
parseNGrams(ArpaLmReaderCallback<ProbBackoffPair> callback)
protected java.lang.String
readLine()
-
-
-
Field Detail
-
START_SYMBOL
public static final java.lang.String START_SYMBOL
- See Also:
- Constant Field Values
-
END_SYMBOL
public static final java.lang.String END_SYMBOL
- See Also:
- Constant Field Values
-
UNK_SYMBOL
public static final java.lang.String UNK_SYMBOL
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
ArpaLmReader
public ArpaLmReader(java.lang.String file, WordIndexer<W> wordIndexer, int maxNgramOrder)
- Parameters:
reader
-
-
-
Method Detail
-
readLine
protected java.lang.String readLine() throws java.io.IOException
- Returns:
- Throws:
java.io.IOException
-
parse
public void parse(ArpaLmReaderCallback<ProbBackoffPair> callback)
Parse the ARPA file and populate the relevant fields of the enclosing ICSILanguageModel- Specified by:
parse
in interfaceLmReader<ProbBackoffPair,ArpaLmReaderCallback<ProbBackoffPair>>
-
parseHeader
protected java.util.List<java.lang.Long> parseHeader()
- Parameters:
callback
-- Throws:
java.io.IOException
ARPAParserException
-
parseNGrams
protected void parseNGrams(ArpaLmReaderCallback<ProbBackoffPair> callback)
-
-