Package org.languagetool.rules.ngrams
Class GoogleToken
java.lang.Object
org.languagetool.rules.ngrams.GoogleToken
A token as tokenized in the Google ngram index.
- Since:
- 3.2
-
Field Summary
Fields -
Constructor Summary
ConstructorsConstructorDescriptionGoogleToken
(String token, int startPos, int endPos) GoogleToken
(String token, int startPos, int endPos, Set<AnalyzedToken> posTags) -
Method Summary
Modifier and TypeMethodDescriptionprivate static Set
<AnalyzedToken> findOriginalAnalyzedTokens
(AnalyzedSentence sentence, int startPos, int endPos) (package private) static List
<GoogleToken> getGoogleTokens
(String sentence, boolean addStartToken, Tokenizer wordTokenizer) (package private) static List
<GoogleToken> getGoogleTokens
(AnalyzedSentence sentence, boolean addStartToken, Tokenizer wordTokenizer) (package private) Set
<AnalyzedToken> (package private) boolean
toString()
-
Field Details
-
token
-
startPos
final int startPos -
endPos
final int endPos -
posTags
-
-
Constructor Details
-
GoogleToken
GoogleToken(String token, int startPos, int endPos) -
GoogleToken
GoogleToken(String token, int startPos, int endPos, Set<AnalyzedToken> posTags)
-
-
Method Details
-
getPosTags
Set<AnalyzedToken> getPosTags() -
isWhitespace
boolean isWhitespace() -
toString
-
getGoogleTokens
static List<GoogleToken> getGoogleTokens(String sentence, boolean addStartToken, Tokenizer wordTokenizer) -
getGoogleTokens
static List<GoogleToken> getGoogleTokens(AnalyzedSentence sentence, boolean addStartToken, Tokenizer wordTokenizer) -
findOriginalAnalyzedTokens
private static Set<AnalyzedToken> findOriginalAnalyzedTokens(AnalyzedSentence sentence, int startPos, int endPos)
-