Class Parser


  • class Parser
    extends java.lang.Object
    A parser that reads input from the given Reader and parses it to produce a Template.
    • Field Detail

      • reader

        private final java.io.LineNumberReader reader
      • resourceName

        private final java.lang.String resourceName
      • c

        private int c
        The invariant of this parser is that c is always the next character of interest. This means that we almost never have to "unget" a character by reading too far. For example, after we parse an integer, c will be the first character after the integer, which is exactly the state we will be in when there are no more digits.

        Sometimes we need to read two characters ahead, and in that case we use pushback.

      • pushback

        private int pushback
        A single character of pushback. If this is not negative, the next() method will return it instead of reading a character.
      • CODE_POINT_TO_OPERATORS

        private static final com.google.common.collect.ImmutableListMultimap<java.lang.Integer,​Parser.Operator> CODE_POINT_TO_OPERATORS
        Maps a code point to the operators that begin with that code point. For example, maps < to LESS and LESS_OR_EQUAL.
      • ASCII_LETTER

        private static final com.google.common.base.CharMatcher ASCII_LETTER
      • ASCII_DIGIT

        private static final com.google.common.base.CharMatcher ASCII_DIGIT
      • ID_CHAR

        private static final com.google.common.base.CharMatcher ID_CHAR
    • Constructor Detail

      • Parser

        Parser​(java.io.Reader reader,
               java.lang.String resourceName,
               Template.ResourceOpener resourceOpener)
        throws java.io.IOException
        Throws:
        java.io.IOException
    • Method Detail

      • parse

        Template parse()
                throws java.io.IOException
        Parse the input completely to produce a Template.

        Parsing happens in two phases. First, we parse a sequence of "tokens", where tokens include entire references such as

            ${x.foo()[23]}
         
        or entire directives such as
            #set ($x = $y + $z)
         
        But tokens do not span complex constructs. For example,
            #if ($x == $y) something #end
         
        is three tokens:
            #if ($x == $y)
            (literal text " something ")
           #end
         

        The second phase then takes the sequence of tokens and constructs a parse tree out of it. Some nodes in the parse tree will be unchanged from the token sequence, such as the

            ${x.foo()[23]}
            #set ($x = $y + $z)
         
        examples above. But a construct such as the #if ... #end mentioned above will become a single IfNode in the parse tree in the second phase.

        The main reason for this approach is that Velocity has two kinds of lexical contexts. At the top level, there can be arbitrary literal text; references like ${x.foo()}; and directives like #if or #set. Inside the parentheses of a directive, however, neither arbitrary text nor directives can appear, but expressions can, so we need to tokenize the inside of

            #if ($x == $a + $b)
         
        as the five tokens "$x", "==", "$a", "+", "$b". Rather than having a classical parser/lexer combination, where the lexer would need to switch between these two modes, we replace the lexer with an ad-hoc parser that is the first phase described above, and we define a simple parser over the resultant tokens that is the second phase.
        Throws:
        java.io.IOException
      • parseTokens

        private com.google.common.collect.ImmutableList<Node> parseTokens()
                                                                   throws java.io.IOException
        Throws:
        java.io.IOException
      • lineNumber

        private int lineNumber()
      • next

        private void next()
                   throws java.io.IOException
        Gets the next character from the reader and assigns it to c. If there are no more characters, sets c to EOF if it is not already.
        Throws:
        java.io.IOException
      • pushback

        private void pushback​(int c1)
        Saves the current character c to be read again, and sets c to the given c1. Suppose the text contains xy and we have just read y. So c == 'y'. Now if we execute pushback('x'), we will have c == 'x' and the next call to next() will set c == 'y'. Subsequent calls to next() will continue reading from reader. So the pushback essentially puts us back in the state we were in before we read y.
      • skipSpace

        private void skipSpace()
                        throws java.io.IOException
        If c is a space character, keeps reading until c is a non-space character or there are no more characters.
        Throws:
        java.io.IOException
      • nextNonSpace

        private void nextNonSpace()
                           throws java.io.IOException
        Gets the next character from the reader, and if it is a space character, keeps reading until a non-space character is found.
        Throws:
        java.io.IOException
      • expect

        private void expect​(char expected)
                     throws java.io.IOException
        Skips any space in the reader, and then throws an exception if the first non-space character found is not the expected one. Sets c to the first character after that expected one.
        Throws:
        java.io.IOException
      • parseNode

        private Node parseNode()
                        throws java.io.IOException
        Parses a single node from the reader, as part of the first parsing phase.
        
         <template> -> <empty> |
                       <directive> <template> |
                       <non-directive> <template>
         
        Throws:
        java.io.IOException
      • parseHashSquare

        private Node parseHashSquare()
                              throws java.io.IOException
        Throws:
        java.io.IOException
      • parseNonDirective

        private Node parseNonDirective()
                                throws java.io.IOException
        Parses a single non-directive node from the reader.
        
         <non-directive> -> <reference> |
                            <text containing neither $ nor #>
         
        Throws:
        java.io.IOException
      • parseDirective

        private Node parseDirective()
                             throws java.io.IOException
        Parses a single directive token from the reader. Directives can be spelled with or without braces, for example #if or #{if}. We omit the brace spelling in the productions here:
        
         <directive> -> <if-token> |
                        <else-token> |
                        <elseif-token> |
                        <end-token> |
                        <foreach-token> |
                        <set-token> |
                        <parse-token> |
                        <macro-token> |
                        <macro-call> |
                        <comment>
         
        Throws:
        java.io.IOException
      • parseIfOrElseIf

        private Node parseIfOrElseIf​(java.lang.String directive)
                              throws java.io.IOException
        Parses the condition following #if or #elseif.
        
         <if-token> -> #if ( <condition> )
         <elseif-token> -> #elseif ( <condition> )
         
        Parameters:
        directive - either "if" or "elseif".
        Throws:
        java.io.IOException
      • parseForEach

        private Node parseForEach()
                           throws java.io.IOException
        Parses a #foreach token from the reader.
        
         <foreach-token> -> #foreach ( $<id> in <expression> )
         
        Throws:
        java.io.IOException
      • parseSet

        private Node parseSet()
                       throws java.io.IOException
        Parses a #set token from the reader.
        
         <set-token> -> #set ( $<id> = <expression>)
         
        Throws:
        java.io.IOException
      • parseParse

        private Node parseParse()
                         throws java.io.IOException
        Parses a #parse token from the reader.
        
         <parse-token> -> #parse ( <string-literal> )
         

        The way this works is inconsistent with Velocity. In Velocity, the #parse directive is evaluated when it is encountered during template evaluation. That means that the argument can be a variable, and it also means that you can use #if to choose whether or not to do the #parse. Neither of those is true in EscapeVelocity. The contents of the #parse are integrated into the containing template pretty much as if they had been written inline. That also means that EscapeVelocity allows forward references to macros inside #parse directives, which Velocity does not.

        Throws:
        java.io.IOException
      • parseMacroDefinition

        private Node parseMacroDefinition()
                                   throws java.io.IOException
        Parses a #macro token from the reader.
        
         <macro-token> -> #macro ( <id> <macro-parameter-list> )
         <macro-parameter-list> -> <empty> |
                                   $<id> <macro-parameter-list>
         

        Macro parameters are not separated by commas, though method-reference parameters are.

        Throws:
        java.io.IOException
      • parsePossibleMacroCall

        private Node parsePossibleMacroCall​(java.lang.String directive)
                                     throws java.io.IOException
        Parses an identifier after # that is not one of the standard directives. The assumption is that it is a call of a macro that is defined in the template. Macro definitions are extracted from the template during the second parsing phase (and not during evaluation of the template as you might expect). This means that a macro can be called before it is defined.
        
         <macro-call> -> # <id> ( <expression-list> )
         <expression-list> -> <empty> |
                              <expression> <optional-comma> <expression-list>
         <optional-comma> -> <empty> | ,
         
        Throws:
        java.io.IOException
      • parseLineComment

        private Node parseLineComment()
                               throws java.io.IOException
        Parses and discards a line comment, which is ## followed by any number of characters up to and including the next newline.
        Throws:
        java.io.IOException
      • parseBlockComment

        private Node parseBlockComment()
                                throws java.io.IOException
        Parses and discards a block comment, which is #* followed by everything up to and including the next *#.
        Throws:
        java.io.IOException
      • parsePlainText

        private Node parsePlainText​(int firstChar)
                             throws java.io.IOException
        Parses plain text, which is text that contains neither $ nor #. The given firstChar is the first character of the plain text, and c is the second (if the plain text is more than one character).
        Throws:
        java.io.IOException
      • parsePlainText

        private Node parsePlainText​(java.lang.StringBuilder sb)
                             throws java.io.IOException
        Throws:
        java.io.IOException
      • parseReference

        private Node parseReference()
                             throws java.io.IOException
        Parses a reference, which is everything that can start with a $. References can optionally be enclosed in braces, so $x and ${x} are the same. Braces are useful when text after the reference would otherwise be parsed as part of it. For example, ${x}y is a reference to the variable $x, followed by the plain text y. Of course $xy would be a reference to the variable $xy.
        
         <reference> -> $<reference-no-brace> |
                        ${<reference-no-brace>}
         

        On entry to this method, c is the character immediately after the $.

        Throws:
        java.io.IOException
      • parseRequiredReference

        private ReferenceNode parseRequiredReference()
                                              throws java.io.IOException
        Same as parseReference(), except it really must be a reference. A $ in normal text doesn't start a reference if it is not followed by an identifier. But in an expression, for example in #if ($x == 23), $ must be followed by an identifier.
        Throws:
        java.io.IOException
      • parseReferenceNoBrace

        private ReferenceNode parseReferenceNoBrace()
                                             throws java.io.IOException
        Parses a reference, in the simple form without braces.
        
         <reference-no-brace> -> <id><reference-suffix>
         
        Throws:
        java.io.IOException
      • parseReferenceSuffix

        private ReferenceNode parseReferenceSuffix​(ReferenceNode lhs)
                                            throws java.io.IOException
        Parses the modifiers that can appear at the tail of a reference.
        
         <reference-suffix> -> <empty> |
                               <reference-member> |
                               <reference-index>
         
        Parameters:
        lhs - the reference node representing the first part of the reference $x in $x.foo or $x.foo(), or later $x.y in $x.y.z.
        Throws:
        java.io.IOException
      • parseReferenceMember

        private ReferenceNode parseReferenceMember​(ReferenceNode lhs)
                                            throws java.io.IOException
        Parses a reference member, which is either a property reference like $x.y or a method call like $x.y($z).
        
         <reference-member> -> .<id><reference-property-or-method><reference-suffix>
         <reference-property-or-method> -> <id> |
                                           <id> ( <method-parameter-list> )
         
        Parameters:
        lhs - the reference node representing what appears to the left of the dot, like the $x in $x.foo or $x.foo().
        Throws:
        java.io.IOException
      • parseReferenceMethodParams

        private ReferenceNode parseReferenceMethodParams​(ReferenceNode lhs,
                                                         java.lang.String id)
                                                  throws java.io.IOException
        Parses the parameters to a method reference, like $foo.bar($a, $b).
        
         <method-parameter-list> -> <empty> |
                                    <non-empty-method-parameter-list>
         <non-empty-method-parameter-list> -> <expression> |
                                              <expression> , <non-empty-method-parameter-list>
         
        Parameters:
        lhs - the reference node representing what appears to the left of the dot, like the $x in $x.foo().
        Throws:
        java.io.IOException
      • parseReferenceIndex

        private ReferenceNode parseReferenceIndex​(ReferenceNode lhs)
                                           throws java.io.IOException
        Parses an index suffix to a method, like $x[$i].
        
         <reference-index> -> [ <expression> ]
         
        Parameters:
        lhs - the reference node representing what appears to the left of the dot, like the $x in $x[$i].
        Throws:
        java.io.IOException
      • parseExpression

        private ExpressionNode parseExpression()
                                        throws java.io.IOException
        Parses an expression, which can occur within a directive like #if or #set, or within a reference like $x[$a + $b] or $x.m($a + $b).
        
         <expression> -> <and-expression> |
                         <expression> || <and-expression>
         <and-expression> -> <relational-expression> |
                             <and-expression> && <relational-expression>
         <equality-exression> -> <relational-expression> |
                                 <equality-expression> <equality-op> <relational-expression>
         <equality-op> -> == | !=
         <relational-expression> -> <additive-expression> |
                                    <relational-expression> <relation> <additive-expression>
         <relation> -> < | <= | > | >=
         <additive-expression> -> <multiplicative-expression> |
                                  <additive-expression> <add-op> <multiplicative-expression>
         <add-op> -> + | -
         <multiplicative-expression> -> <unary-expression> |
                                        <multiplicative-expression> <mult-op> <unary-expression>
         <mult-op> -> * | / | %
         
        Throws:
        java.io.IOException
      • parseUnaryExpression

        private ExpressionNode parseUnaryExpression()
                                             throws java.io.IOException
        Parses an expression not containing any operators (except inside parentheses).
        
         <unary-expression> -> <primary> |
                               ( <expression> ) |
                               ! <unary-expression>
         
        Throws:
        java.io.IOException
      • parsePrimary

        private ExpressionNode parsePrimary()
                                     throws java.io.IOException
        Parses an expression containing only literals or references.
        
         <primary> -> <reference> |
                      <string-literal> |
                      <integer-literal> |
                      <boolean-literal>
         
        Throws:
        java.io.IOException
      • parseStringLiteral

        private ExpressionNode parseStringLiteral()
                                           throws java.io.IOException
        Throws:
        java.io.IOException
      • readStringLiteral

        private java.lang.String readStringLiteral()
                                            throws java.io.IOException
        Throws:
        java.io.IOException
      • parseIntLiteral

        private ExpressionNode parseIntLiteral​(java.lang.String prefix)
                                        throws java.io.IOException
        Throws:
        java.io.IOException
      • parseBooleanLiteral

        private ExpressionNode parseBooleanLiteral()
                                            throws java.io.IOException
        Parses a boolean literal, either true or false. -> true | false
        Throws:
        java.io.IOException
      • isAsciiLetter

        private static boolean isAsciiLetter​(int c)
      • isAsciiDigit

        private static boolean isAsciiDigit​(int c)
      • isIdChar

        private static boolean isIdChar​(int c)
      • parseId

        private java.lang.String parseId​(java.lang.String what)
                                  throws java.io.IOException
        Parse an identifier as specified by the VTL . Identifiers are ASCII: starts with a letter, then letters, digits, - and _.
        Throws:
        java.io.IOException
      • parseException

        private ParseException parseException​(java.lang.String message)
                                       throws java.io.IOException
        Returns an exception to be thrown describing a parse error with the given message, and including information about where it occurred.
        Throws:
        java.io.IOException