Package org.languagetool.languagemodel
Class LuceneLanguageModel
- java.lang.Object
-
- org.languagetool.languagemodel.BaseLanguageModel
-
- org.languagetool.languagemodel.LuceneLanguageModel
-
- All Implemented Interfaces:
java.lang.AutoCloseable
,LanguageModel
public class LuceneLanguageModel extends BaseLanguageModel
LikeLuceneSingleIndexLanguageModel
, but can merge the results of lookups in several independent indexes to one result.- Since:
- 2.7
-
-
Field Summary
Fields Modifier and Type Field Description private java.util.List<LuceneSingleIndexLanguageModel>
lms
-
Fields inherited from interface org.languagetool.languagemodel.LanguageModel
GOOGLE_SENTENCE_END, GOOGLE_SENTENCE_START
-
-
Constructor Summary
Constructors Constructor Description LuceneLanguageModel(java.io.File topIndexDir)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description void
close()
long
getCount(java.lang.String token)
Get the occurrence count fortoken
.long
getCount(java.util.List<java.lang.String> tokens)
Get the occurrence count for the given token sequence.private static @Nullable java.io.File[]
getSubDirectoriesOrNull(java.io.File topIndexDir)
long
getTotalTokenCount()
java.lang.String
toString()
static void
validateDirectory(java.io.File topIndexDir)
-
Methods inherited from class org.languagetool.languagemodel.BaseLanguageModel
getPseudoProbability, getPseudoProbabilityStupidBackoff
-
-
-
-
Field Detail
-
lms
private final java.util.List<LuceneSingleIndexLanguageModel> lms
-
-
Constructor Detail
-
LuceneLanguageModel
public LuceneLanguageModel(java.io.File topIndexDir)
- Parameters:
topIndexDir
- a directory which contains either: 1) sub directories called1grams
,2grams
,3grams
, which are Lucene indexes with ngram occurrences as created byorg.languagetool.dev.FrequencyIndexCreator
or 2) sub directoriesindex-1
,index-2
etc that contain the sub directories described under 1)
-
-
Method Detail
-
validateDirectory
public static void validateDirectory(java.io.File topIndexDir)
-
getSubDirectoriesOrNull
@Nullable private static @Nullable java.io.File[] getSubDirectoriesOrNull(java.io.File topIndexDir)
-
getCount
public long getCount(java.util.List<java.lang.String> tokens)
Description copied from class:BaseLanguageModel
Get the occurrence count for the given token sequence.- Specified by:
getCount
in classBaseLanguageModel
-
getCount
public long getCount(java.lang.String token)
Description copied from class:BaseLanguageModel
Get the occurrence count fortoken
.- Specified by:
getCount
in classBaseLanguageModel
-
getTotalTokenCount
public long getTotalTokenCount()
- Specified by:
getTotalTokenCount
in classBaseLanguageModel
-
close
public void close()
-
toString
public java.lang.String toString()
- Overrides:
toString
in classjava.lang.Object
-
-