Class Parser
- java.lang.Object
-
- com.google.auto.value.processor.escapevelocity.Parser
-
class Parser extends java.lang.Object
A parser that reads input from the givenReader
and parses it to produce aTemplate
.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description (package private) static class
Parser.Operator
private class
Parser.OperatorParser
An operator-precedence parser for the binary operations we understand.
-
Field Summary
Fields Modifier and Type Field Description private static com.google.common.base.CharMatcher
ASCII_DIGIT
private static com.google.common.base.CharMatcher
ASCII_LETTER
private int
c
The invariant of this parser is thatc
is always the next character of interest.private static com.google.common.collect.ImmutableListMultimap<java.lang.Integer,Parser.Operator>
CODE_POINT_TO_OPERATORS
Maps a code point to the operators that begin with that code point.private static int
EOF
private static com.google.common.base.CharMatcher
ID_CHAR
private int
pushback
A single character of pushback.private java.io.LineNumberReader
reader
private java.lang.String
resourceName
private Template.ResourceOpener
resourceOpener
-
Constructor Summary
Constructors Constructor Description Parser(java.io.Reader reader, java.lang.String resourceName, Template.ResourceOpener resourceOpener)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description private void
expect(char expected)
Skips any space in the reader, and then throws an exception if the first non-space character found is not the expected one.private static boolean
isAsciiDigit(int c)
private static boolean
isAsciiLetter(int c)
private static boolean
isIdChar(int c)
private int
lineNumber()
private void
next()
Gets the next character from the reader and assigns it toc
.private void
nextNonSpace()
Gets the next character from the reader, and if it is a space character, keeps reading until a non-space character is found.(package private) Template
parse()
Parse the input completely to produce aTemplate
.private Node
parseBlockComment()
Parses and discards a block comment, which is#*
followed by everything up to and including the next*#
.private ExpressionNode
parseBooleanLiteral()
Parses a boolean literal, eithertrue
orfalse
.private Node
parseDirective()
Parses a single directive token from the reader.private ParseException
parseException(java.lang.String message)
Returns an exception to be thrown describing a parse error with the given message, and including information about where it occurred.private ExpressionNode
parseExpression()
Parses an expression, which can occur within a directive like#if
or#set
, or within a reference like$x[$a + $b]
or$x.m($a + $b)
.private Node
parseForEach()
Parses a#foreach
token from the reader.private Node
parseHashSquare()
private java.lang.String
parseId(java.lang.String what)
Parse an identifier as specified by the VTL .private Node
parseIfOrElseIf(java.lang.String directive)
Parses the condition following#if
or#elseif
.private ExpressionNode
parseIntLiteral(java.lang.String prefix)
private Node
parseLineComment()
Parses and discards a line comment, which is##
followed by any number of characters up to and including the next newline.private Node
parseMacroDefinition()
Parses a#macro
token from the reader.private Node
parseNode()
Parses a single node from the reader, as part of the first parsing phase.private Node
parseNonDirective()
Parses a single non-directive node from the reader.private Node
parseParse()
Parses a#parse
token from the reader.private Node
parsePlainText(int firstChar)
Parses plain text, which is text that contains neither$
nor#
.private Node
parsePlainText(java.lang.StringBuilder sb)
private Node
parsePossibleMacroCall(java.lang.String directive)
Parses an identifier after#
that is not one of the standard directives.private ExpressionNode
parsePrimary()
Parses an expression containing only literals or references.private Node
parseReference()
Parses a reference, which is everything that can start with a$
.private ReferenceNode
parseReferenceIndex(ReferenceNode lhs)
Parses an index suffix to a method, like$x[$i]
.private ReferenceNode
parseReferenceMember(ReferenceNode lhs)
Parses a reference member, which is either a property reference like$x.y
or a method call like$x.y($z)
.private ReferenceNode
parseReferenceMethodParams(ReferenceNode lhs, java.lang.String id)
Parses the parameters to a method reference, like$foo.bar($a, $b)
.private ReferenceNode
parseReferenceNoBrace()
Parses a reference, in the simple form without braces.private ReferenceNode
parseReferenceSuffix(ReferenceNode lhs)
Parses the modifiers that can appear at the tail of a reference.private ReferenceNode
parseRequiredReference()
Same asparseReference()
, except it really must be a reference.private Node
parseSet()
Parses a#set
token from the reader.private ExpressionNode
parseStringLiteral()
private com.google.common.collect.ImmutableList<Node>
parseTokens()
private ExpressionNode
parseUnaryExpression()
Parses an expression not containing any operators (except inside parentheses).private void
pushback(int c1)
Saves the current characterc
to be read again, and setsc
to the givenc1
.private java.lang.String
readStringLiteral()
private void
skipSpace()
Ifc
is a space character, keeps reading untilc
is a non-space character or there are no more characters.
-
-
-
Field Detail
-
EOF
private static final int EOF
- See Also:
- Constant Field Values
-
reader
private final java.io.LineNumberReader reader
-
resourceName
private final java.lang.String resourceName
-
resourceOpener
private final Template.ResourceOpener resourceOpener
-
c
private int c
The invariant of this parser is thatc
is always the next character of interest. This means that we almost never have to "unget" a character by reading too far. For example, after we parse an integer,c
will be the first character after the integer, which is exactly the state we will be in when there are no more digits.Sometimes we need to read two characters ahead, and in that case we use
pushback
.
-
pushback
private int pushback
A single character of pushback. If this is not negative, thenext()
method will return it instead of reading a character.
-
CODE_POINT_TO_OPERATORS
private static final com.google.common.collect.ImmutableListMultimap<java.lang.Integer,Parser.Operator> CODE_POINT_TO_OPERATORS
Maps a code point to the operators that begin with that code point. For example, maps<
toLESS
andLESS_OR_EQUAL
.
-
ASCII_LETTER
private static final com.google.common.base.CharMatcher ASCII_LETTER
-
ASCII_DIGIT
private static final com.google.common.base.CharMatcher ASCII_DIGIT
-
ID_CHAR
private static final com.google.common.base.CharMatcher ID_CHAR
-
-
Constructor Detail
-
Parser
Parser(java.io.Reader reader, java.lang.String resourceName, Template.ResourceOpener resourceOpener) throws java.io.IOException
- Throws:
java.io.IOException
-
-
Method Detail
-
parse
Template parse() throws java.io.IOException
Parse the input completely to produce aTemplate
.Parsing happens in two phases. First, we parse a sequence of "tokens", where tokens include entire references such as
${x.foo()[23]}
or entire directives such as#set ($x = $y + $z)
But tokens do not span complex constructs. For example,#if ($x == $y) something #end
is three tokens:#if ($x == $y) (literal text " something ") #end
The second phase then takes the sequence of tokens and constructs a parse tree out of it. Some nodes in the parse tree will be unchanged from the token sequence, such as the
${x.foo()[23]} #set ($x = $y + $z)
examples above. But a construct such as the#if ... #end
mentioned above will become a single IfNode in the parse tree in the second phase.The main reason for this approach is that Velocity has two kinds of lexical contexts. At the top level, there can be arbitrary literal text; references like
${x.foo()}
; and directives like#if
or#set
. Inside the parentheses of a directive, however, neither arbitrary text nor directives can appear, but expressions can, so we need to tokenize the inside of#if ($x == $a + $b)
as the five tokens "$x", "==", "$a", "+", "$b". Rather than having a classical parser/lexer combination, where the lexer would need to switch between these two modes, we replace the lexer with an ad-hoc parser that is the first phase described above, and we define a simple parser over the resultant tokens that is the second phase.- Throws:
java.io.IOException
-
parseTokens
private com.google.common.collect.ImmutableList<Node> parseTokens() throws java.io.IOException
- Throws:
java.io.IOException
-
lineNumber
private int lineNumber()
-
next
private void next() throws java.io.IOException
Gets the next character from the reader and assigns it toc
. If there are no more characters, setsc
toEOF
if it is not already.- Throws:
java.io.IOException
-
pushback
private void pushback(int c1)
Saves the current characterc
to be read again, and setsc
to the givenc1
. Suppose the text containsxy
and we have just ready
. Soc == 'y'
. Now if we executepushback('x')
, we will havec == 'x'
and the next call tonext()
will setc == 'y'
. Subsequent calls tonext()
will continue reading fromreader
. So the pushback essentially puts us back in the state we were in before we ready
.
-
skipSpace
private void skipSpace() throws java.io.IOException
Ifc
is a space character, keeps reading untilc
is a non-space character or there are no more characters.- Throws:
java.io.IOException
-
nextNonSpace
private void nextNonSpace() throws java.io.IOException
Gets the next character from the reader, and if it is a space character, keeps reading until a non-space character is found.- Throws:
java.io.IOException
-
expect
private void expect(char expected) throws java.io.IOException
Skips any space in the reader, and then throws an exception if the first non-space character found is not the expected one. Setsc
to the first character after that expected one.- Throws:
java.io.IOException
-
parseNode
private Node parseNode() throws java.io.IOException
Parses a single node from the reader, as part of the first parsing phase.<template> -> <empty> | <directive> <template> | <non-directive> <template>
- Throws:
java.io.IOException
-
parseHashSquare
private Node parseHashSquare() throws java.io.IOException
- Throws:
java.io.IOException
-
parseNonDirective
private Node parseNonDirective() throws java.io.IOException
Parses a single non-directive node from the reader.<non-directive> -> <reference> | <text containing neither $ nor #>
- Throws:
java.io.IOException
-
parseDirective
private Node parseDirective() throws java.io.IOException
Parses a single directive token from the reader. Directives can be spelled with or without braces, for example#if
or#{if}
. We omit the brace spelling in the productions here:<directive> -> <if-token> | <else-token> | <elseif-token> | <end-token> | <foreach-token> | <set-token> | <parse-token> | <macro-token> | <macro-call> | <comment>
- Throws:
java.io.IOException
-
parseIfOrElseIf
private Node parseIfOrElseIf(java.lang.String directive) throws java.io.IOException
Parses the condition following#if
or#elseif
.<if-token> -> #if ( <condition> ) <elseif-token> -> #elseif ( <condition> )
- Parameters:
directive
- either"if"
or"elseif"
.- Throws:
java.io.IOException
-
parseForEach
private Node parseForEach() throws java.io.IOException
Parses a#foreach
token from the reader.<foreach-token> -> #foreach ( $<id> in <expression> )
- Throws:
java.io.IOException
-
parseSet
private Node parseSet() throws java.io.IOException
Parses a#set
token from the reader.<set-token> -> #set ( $<id> = <expression>)
- Throws:
java.io.IOException
-
parseParse
private Node parseParse() throws java.io.IOException
Parses a#parse
token from the reader.<parse-token> -> #parse ( <string-literal> )
The way this works is inconsistent with Velocity. In Velocity, the
#parse
directive is evaluated when it is encountered during template evaluation. That means that the argument can be a variable, and it also means that you can use#if
to choose whether or not to do the#parse
. Neither of those is true in EscapeVelocity. The contents of the#parse
are integrated into the containing template pretty much as if they had been written inline. That also means that EscapeVelocity allows forward references to macros inside#parse
directives, which Velocity does not.- Throws:
java.io.IOException
-
parseMacroDefinition
private Node parseMacroDefinition() throws java.io.IOException
Parses a#macro
token from the reader.<macro-token> -> #macro ( <id> <macro-parameter-list> ) <macro-parameter-list> -> <empty> | $<id> <macro-parameter-list>
Macro parameters are not separated by commas, though method-reference parameters are.
- Throws:
java.io.IOException
-
parsePossibleMacroCall
private Node parsePossibleMacroCall(java.lang.String directive) throws java.io.IOException
Parses an identifier after#
that is not one of the standard directives. The assumption is that it is a call of a macro that is defined in the template. Macro definitions are extracted from the template during the second parsing phase (and not during evaluation of the template as you might expect). This means that a macro can be called before it is defined.<macro-call> -> # <id> ( <expression-list> ) <expression-list> -> <empty> | <expression> <optional-comma> <expression-list> <optional-comma> -> <empty> | ,
- Throws:
java.io.IOException
-
parseLineComment
private Node parseLineComment() throws java.io.IOException
Parses and discards a line comment, which is##
followed by any number of characters up to and including the next newline.- Throws:
java.io.IOException
-
parseBlockComment
private Node parseBlockComment() throws java.io.IOException
Parses and discards a block comment, which is#*
followed by everything up to and including the next*#
.- Throws:
java.io.IOException
-
parsePlainText
private Node parsePlainText(int firstChar) throws java.io.IOException
Parses plain text, which is text that contains neither$
nor#
. The givenfirstChar
is the first character of the plain text, andc
is the second (if the plain text is more than one character).- Throws:
java.io.IOException
-
parsePlainText
private Node parsePlainText(java.lang.StringBuilder sb) throws java.io.IOException
- Throws:
java.io.IOException
-
parseReference
private Node parseReference() throws java.io.IOException
Parses a reference, which is everything that can start with a$
. References can optionally be enclosed in braces, so$x
and${x}
are the same. Braces are useful when text after the reference would otherwise be parsed as part of it. For example,${x}y
is a reference to the variable$x
, followed by the plain texty
. Of course$xy
would be a reference to the variable$xy
.<reference> -> $<reference-no-brace> | ${<reference-no-brace>}
On entry to this method,
c
is the character immediately after the$
.- Throws:
java.io.IOException
-
parseRequiredReference
private ReferenceNode parseRequiredReference() throws java.io.IOException
Same asparseReference()
, except it really must be a reference. A$
in normal text doesn't start a reference if it is not followed by an identifier. But in an expression, for example in#if ($x == 23)
,$
must be followed by an identifier.- Throws:
java.io.IOException
-
parseReferenceNoBrace
private ReferenceNode parseReferenceNoBrace() throws java.io.IOException
Parses a reference, in the simple form without braces.<reference-no-brace> -> <id><reference-suffix>
- Throws:
java.io.IOException
-
parseReferenceSuffix
private ReferenceNode parseReferenceSuffix(ReferenceNode lhs) throws java.io.IOException
Parses the modifiers that can appear at the tail of a reference.<reference-suffix> -> <empty> | <reference-member> | <reference-index>
- Parameters:
lhs
- the reference node representing the first part of the reference$x
in$x.foo
or$x.foo()
, or later$x.y
in$x.y.z
.- Throws:
java.io.IOException
-
parseReferenceMember
private ReferenceNode parseReferenceMember(ReferenceNode lhs) throws java.io.IOException
Parses a reference member, which is either a property reference like$x.y
or a method call like$x.y($z)
.<reference-member> -> .<id><reference-property-or-method><reference-suffix> <reference-property-or-method> -> <id> | <id> ( <method-parameter-list> )
- Parameters:
lhs
- the reference node representing what appears to the left of the dot, like the$x
in$x.foo
or$x.foo()
.- Throws:
java.io.IOException
-
parseReferenceMethodParams
private ReferenceNode parseReferenceMethodParams(ReferenceNode lhs, java.lang.String id) throws java.io.IOException
Parses the parameters to a method reference, like$foo.bar($a, $b)
.<method-parameter-list> -> <empty> | <non-empty-method-parameter-list> <non-empty-method-parameter-list> -> <expression> | <expression> , <non-empty-method-parameter-list>
- Parameters:
lhs
- the reference node representing what appears to the left of the dot, like the$x
in$x.foo()
.- Throws:
java.io.IOException
-
parseReferenceIndex
private ReferenceNode parseReferenceIndex(ReferenceNode lhs) throws java.io.IOException
Parses an index suffix to a method, like$x[$i]
.<reference-index> -> [ <expression> ]
- Parameters:
lhs
- the reference node representing what appears to the left of the dot, like the$x
in$x[$i]
.- Throws:
java.io.IOException
-
parseExpression
private ExpressionNode parseExpression() throws java.io.IOException
Parses an expression, which can occur within a directive like#if
or#set
, or within a reference like$x[$a + $b]
or$x.m($a + $b)
.<expression> -> <and-expression> | <expression> || <and-expression> <and-expression> -> <relational-expression> | <and-expression> && <relational-expression> <equality-exression> -> <relational-expression> | <equality-expression> <equality-op> <relational-expression> <equality-op> -> == | != <relational-expression> -> <additive-expression> | <relational-expression> <relation> <additive-expression> <relation> -> < | <= | > | >= <additive-expression> -> <multiplicative-expression> | <additive-expression> <add-op> <multiplicative-expression> <add-op> -> + | - <multiplicative-expression> -> <unary-expression> | <multiplicative-expression> <mult-op> <unary-expression> <mult-op> -> * | / | %
- Throws:
java.io.IOException
-
parseUnaryExpression
private ExpressionNode parseUnaryExpression() throws java.io.IOException
Parses an expression not containing any operators (except inside parentheses).<unary-expression> -> <primary> | ( <expression> ) | ! <unary-expression>
- Throws:
java.io.IOException
-
parsePrimary
private ExpressionNode parsePrimary() throws java.io.IOException
Parses an expression containing only literals or references.<primary> -> <reference> | <string-literal> | <integer-literal> | <boolean-literal>
- Throws:
java.io.IOException
-
parseStringLiteral
private ExpressionNode parseStringLiteral() throws java.io.IOException
- Throws:
java.io.IOException
-
readStringLiteral
private java.lang.String readStringLiteral() throws java.io.IOException
- Throws:
java.io.IOException
-
parseIntLiteral
private ExpressionNode parseIntLiteral(java.lang.String prefix) throws java.io.IOException
- Throws:
java.io.IOException
-
parseBooleanLiteral
private ExpressionNode parseBooleanLiteral() throws java.io.IOException
Parses a boolean literal, eithertrue
orfalse
.-> true | false - Throws:
java.io.IOException
-
isAsciiLetter
private static boolean isAsciiLetter(int c)
-
isAsciiDigit
private static boolean isAsciiDigit(int c)
-
isIdChar
private static boolean isIdChar(int c)
-
parseId
private java.lang.String parseId(java.lang.String what) throws java.io.IOException
Parse an identifier as specified by the VTL . Identifiers are ASCII: starts with a letter, then letters, digits,-
and_
.- Throws:
java.io.IOException
-
parseException
private ParseException parseException(java.lang.String message) throws java.io.IOException
Returns an exception to be thrown describing a parse error with the given message, and including information about where it occurred.- Throws:
java.io.IOException
-
-