Package edu.umd.cs.findbugs
Class Tokenizer
- java.lang.Object
-
- edu.umd.cs.findbugs.Tokenizer
-
public class Tokenizer extends java.lang.Object
A simple tokenizer for Java source text. This is not intended to be a compliant lexer; instead, it is for quick and dirty scanning.- See Also:
Token
-
-
Field Summary
Fields Modifier and Type Field Description private java.io.PushbackReader
reader
private static java.util.BitSet
single
private static java.util.BitSet
whiteSpace
-
Constructor Summary
Constructors Constructor Description Tokenizer(java.io.Reader reader)
Constructor.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description private Token
maybeComment()
private Token
munchString(int delimiter)
Token
next()
Get the next Token in the stream.private Token
parseWord()
private void
skipWhitespace()
-
-
-
Method Detail
-
next
public Token next() throws java.io.IOException
Get the next Token in the stream.- Returns:
- the Token
- Throws:
java.io.IOException
-
skipWhitespace
private void skipWhitespace() throws java.io.IOException
- Throws:
java.io.IOException
-
munchString
private Token munchString(int delimiter) throws java.io.IOException
- Throws:
java.io.IOException
-
maybeComment
private Token maybeComment() throws java.io.IOException
- Throws:
java.io.IOException
-
parseWord
private Token parseWord() throws java.io.IOException
- Throws:
java.io.IOException
-
-