Package gnu.text
Class Lexer
- java.lang.Object
-
- java.io.Reader
-
- gnu.text.Lexer
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
,Readable
- Direct Known Subclasses:
Lexer
,LispReader
,XQParser
,XslTranslator
public class Lexer extends Reader
Framework for implementing lexical scanners and parsers.
-
-
Field Summary
Fields Modifier and Type Field Description protected int
nesting
protected gnu.kawa.io.InPort
port
protected boolean
tentative
char[]
tokenBuffer
For building tokens of various kinds.int
tokenBufferLength
The number of chars of tokenBuffer that are used.
-
Constructor Summary
Constructors Constructor Description Lexer(gnu.kawa.io.InPort port)
Lexer(gnu.kawa.io.InPort port, SourceMessages messages)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description boolean
checkErrors(PrintWriter out, int max)
Returns true if any error were seen.boolean
checkNext(char ch)
Check if the next character matches a given character.void
clearErrors()
void
close()
void
eofError(String msg)
void
eofError(String message, int startLine, int startColumn)
void
error(char severity, String message)
void
error(char severity, String filename, int line, int column, String message)
void
error(String message)
void
fatal(String message)
int
getColumnNumber()
Return the current (zero-based) column number.SourceError
getErrors()
int
getLineNumber()
Get the current line number.SourceMessages
getMessages()
String
getName()
gnu.kawa.io.InPort
getPort()
boolean
isInteractive()
boolean
isTentative()
True if input may be incomplete or actively edited.void
mark()
Start tentative parsing.int
peek()
void
popNesting(char save)
Exit a nested expression, reversing pushNestingchar
pushNesting(char promptChar)
Enter a nested expression.int
read()
int
read(char[] buf, int offset, int length)
int
readCodePoint()
Read a Unicode character (codepoint) by checking for surrogates.boolean
readDelimited(String delimiter)
Scan until a given delimiter.static long
readDigits(gnu.kawa.io.InPort port, int radix)
static long
readDigitsInBuffer(gnu.kawa.io.InPort port, long ival, int radix)
Read digits, up to the first non-digit or the buffer limitint
readIntDigits()
int
readOptionalExponent()
Read an optional signed integer.int
readUnicodeChar()
Deprecated.UsereadCodePoint()
.void
reset()
Stop tentative parsing.boolean
seenErrors()
void
setInteractive(boolean v)
void
setMessages(SourceMessages messages)
void
setTentative(boolean v)
void
skip()
protected void
skip_quick()
void
tokenBufferAppend(int ch)
Append one character to tokenBuffer, resizing it if need be.String
tokenBufferString()
protected void
unread()
void
unread(int ch)
protected void
unread_quick()
-
Methods inherited from class java.io.Reader
mark, markSupported, nullReader, read, read, ready, skip, transferTo
-
-
-
-
Constructor Detail
-
Lexer
public Lexer(gnu.kawa.io.InPort port)
-
Lexer
public Lexer(gnu.kawa.io.InPort port, SourceMessages messages)
-
-
Method Detail
-
pushNesting
public char pushNesting(char promptChar)
Enter a nested expression. This is used in interactive mode to control whether to continue past end of line, depending on whether the expression is incomplete.- Parameters:
promptChar
- Used in prompt string to indicate type of nesting.- Returns:
- The previous value of promptChar, to be passed to popNesting.
-
popNesting
public void popNesting(char save)
Exit a nested expression, reversing pushNesting- Parameters:
save
- Saved values return by prior pushNesting
-
getPort
public final gnu.kawa.io.InPort getPort()
-
close
public void close() throws IOException
- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceCloseable
- Specified by:
close
in classReader
- Throws:
IOException
-
read
public int read() throws IOException
- Overrides:
read
in classReader
- Throws:
IOException
-
readUnicodeChar
@Deprecated public int readUnicodeChar() throws IOException
Deprecated.UsereadCodePoint()
.Read a Unicode character (codepoint) by checking for surrogates.- Throws:
IOException
-
readCodePoint
public int readCodePoint() throws IOException
Read a Unicode character (codepoint) by checking for surrogates.- Throws:
IOException
-
read
public int read(char[] buf, int offset, int length) throws IOException
- Specified by:
read
in classReader
- Throws:
IOException
-
unread
public void unread(int ch) throws IOException
- Throws:
IOException
-
peek
public int peek() throws IOException
- Throws:
IOException
-
skip
public void skip() throws IOException
- Throws:
IOException
-
unread
protected void unread() throws IOException
- Throws:
IOException
-
unread_quick
protected void unread_quick() throws IOException
- Throws:
IOException
-
checkNext
public boolean checkNext(char ch) throws IOException
Check if the next character matches a given character.- Parameters:
ch
- The character to match against.- Returns:
- if the character read matches On a match, the position is advanced following that character.
- Throws:
IOException
-
skip_quick
protected void skip_quick() throws IOException
- Throws:
IOException
-
getMessages
public SourceMessages getMessages()
-
setMessages
public void setMessages(SourceMessages messages)
-
checkErrors
public boolean checkErrors(PrintWriter out, int max)
Returns true if any error were seen. Prints and clears the errors.- Parameters:
out
- where to write the error message tomax
- maximum number of messages to print (can be 0)
-
getErrors
public SourceError getErrors()
-
seenErrors
public boolean seenErrors()
-
clearErrors
public void clearErrors()
-
error
public void error(char severity, String message)
-
error
public void error(String message)
-
fatal
public void fatal(String message) throws SyntaxException
- Throws:
SyntaxException
-
eofError
public void eofError(String msg) throws SyntaxException
- Throws:
SyntaxException
-
eofError
public void eofError(String message, int startLine, int startColumn) throws SyntaxException
- Throws:
SyntaxException
-
readOptionalExponent
public int readOptionalExponent() throws IOException
Read an optional signed integer. If there is no integer in the input stream, return 1. For excessively large exponents, return Integer.MIN_VALUE or Integer.MAX_VALUE.- Throws:
IOException
-
readDelimited
public boolean readDelimited(String delimiter) throws IOException, SyntaxException
Scan until a given delimiter. On success, text upto the delimiter is in then tokenBuffer (with tokenBufferLength marking its length); the delimiter is not included.- Throws:
IOException
SyntaxException
-
readDigitsInBuffer
public static long readDigitsInBuffer(gnu.kawa.io.InPort port, long ival, int radix)
Read digits, up to the first non-digit or the buffer limit- Parameters:
ival
- previously-seen digits or -2 if no digits seen- Returns:
- the digits seen as a non-negative long, or -1 on overflow, or -2 if no digits seen
-
readDigits
public static long readDigits(gnu.kawa.io.InPort port, int radix) throws IOException
- Throws:
IOException
-
readIntDigits
public int readIntDigits() throws IOException
- Throws:
IOException
-
getName
public String getName()
-
getLineNumber
public int getLineNumber()
Get the current line number. The "first" line is number number 0.
-
getColumnNumber
public int getColumnNumber()
Return the current (zero-based) column number.
-
isInteractive
public boolean isInteractive()
-
setInteractive
public void setInteractive(boolean v)
-
isTentative
public boolean isTentative()
True if input may be incomplete or actively edited. Used for command-completion and on-the-fly error checking.
-
setTentative
public void setTentative(boolean v)
-
tokenBufferAppend
public void tokenBufferAppend(int ch)
Append one character to tokenBuffer, resizing it if need be.
-
tokenBufferString
public String tokenBufferString()
-
mark
public void mark() throws IOException
Start tentative parsing. Must be followed by a reset.- Throws:
IOException
-
reset
public void reset() throws IOException
Stop tentative parsing. Return to position where we called mark.- Overrides:
reset
in classReader
- Throws:
IOException
-
-