Package org.languagetool.tokenizers.ml
Class MalayalamWordTokenizer
- java.lang.Object
-
- org.languagetool.tokenizers.ml.MalayalamWordTokenizer
-
- All Implemented Interfaces:
org.languagetool.tokenizers.Tokenizer
public class MalayalamWordTokenizer extends java.lang.Object implements org.languagetool.tokenizers.Tokenizer
Tokenizes a sentence into words. Punctuation and whitespace gets its own token.
-
-
Constructor Summary
Constructors Constructor Description MalayalamWordTokenizer()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description java.util.List<java.lang.String>
tokenize(java.lang.String text)
-