Class ExpressionParser

  • Direct Known Subclasses:
    PatternParser, QueryParser

    public class ExpressionParser
    extends java.lang.Object
    Parser for XPath expressions and XSLT patterns.

    This code was originally inspired by James Clark's xt but has been totally rewritten (several times)

    The base class handles parsing of XPath 2.0 and XPath 3.0 syntax (switched by a languageVersion variable). Subclasses refine this to handle XQuery syntax (1.0 and 3.0) and XQuery Update syntax.

    Author:
    Michael Kay
    • Constructor Detail

      • ExpressionParser

        public ExpressionParser()
        Create an expression parser
    • Method Detail

      • setCodeInjector

        public void setCodeInjector​(CodeInjector injector)
        Set a CodeInjector which can be used to modify or wrap expressions on the tree as the expression is parsed and the tree is constructed. This is typically used to add tracing code.
        Parameters:
        injector - the code injector to be used
      • getCodeInjector

        public CodeInjector getCodeInjector()
        Set a CodeInjector which can be used to modify or wrap expressions on the tree as the expression is parsed and the tree is constructed. This is typically used to add tracing code.
        Returns:
        the code injector in use, if any; or null otherwise
      • getTokenizer

        public Tokenizer getTokenizer()
        Get the tokenizer (the lexical analyzer)
        Returns:
        the tokenizer (the lexical analyzer)
      • getStaticContext

        public StaticContext getStaticContext()
        Get the static context used by this expression parser
        Returns:
        the static context
      • setDefaultContainer

        public void setDefaultContainer​(Container container)
        Set the default container for newly constructed expressions
        Parameters:
        container - the default container
      • getDefaultContainer

        public Container getDefaultContainer()
        Get the default container for newly constructed expressions
        Returns:
        the default container
      • getNameChecker

        public NameChecker getNameChecker()
        Get the name checker used by this parser
        Returns:
        the name checker (specific to XML 1.0o
      • nextToken

        public void nextToken()
                       throws XPathException
        Read the next token, catching any exception thrown by the tokenizer
        Throws:
        XPathException - if an invalid token is found
      • expect

        public void expect​(int token)
                    throws XPathException
        Expect a given token; fail if the current token is different. Note that this method does not read any tokens.
        Parameters:
        token - the expected token
        Throws:
        XPathException - if the current token is not the expected token
      • grumble

        public void grumble​(java.lang.String message)
                     throws XPathException
        Report a syntax error (a static error with error code XP0003)
        Parameters:
        message - the error message
        Throws:
        XPathException - always thrown: an exception containing the supplied message
      • grumble

        public void grumble​(java.lang.String message,
                            java.lang.String errorCode)
                     throws XPathException
        Report a static error
        Parameters:
        message - the error message
        errorCode - the error code
        Throws:
        XPathException - always thrown: an exception containing the supplied message
      • grumble

        public void grumble​(java.lang.String message,
                            java.lang.String errorCode,
                            int offset)
                     throws XPathException
        Report a static error, with location information
        Parameters:
        message - the error message
        errorCode - the error code
        offset - the coded location of the error, or -1 if the location of the current token should be used
        Throws:
        XPathException - always thrown: an exception containing the supplied message
      • grumble

        protected void grumble​(java.lang.String message,
                               StructuredQName errorCode,
                               int offset)
                        throws XPathException
        Report a static error
        Parameters:
        message - the error message
        errorCode - the error code
        offset - the coded location of the error, or -1 if the location of the current token should be used
        Throws:
        XPathException - always thrown: an exception containing the supplied message
      • warning

        protected void warning​(java.lang.String message)
                        throws XPathException
        Output a warning message
        Parameters:
        message - the text of the message
        Throws:
        XPathException - if the message cannot be output
      • setLanguage

        public void setLanguage​(int language,
                                DecimalValue version)
        Set the current language (XPath or XQuery, XSLT Pattern, or SequenceType)
        Parameters:
        language - one of the constants XPATH, XQUERY, XSLT_PATTERN, SEQUENCE_TYPE
        version - The XPath or XQuery language version. For XQuery the value must be "1.0" or "3.0; for XPath it must be "2.0" or "3.0". Currently support for XQuery 3.0 and XPath 3.0 is incomplete: check the release notes.
      • getLanguage

        protected java.lang.String getLanguage()
        Get the current language (XPath or XQuery)
        Returns:
        a string representation of the language being parsed, for use in error messages
      • currentTokenDisplay

        protected java.lang.String currentTokenDisplay()
        Display the current token in an error message
        Returns:
        the display representation of the token
      • parse

        public Expression parse​(java.lang.String expression,
                                int start,
                                int terminator,
                                int lineNumber,
                                StaticContext env)
                         throws XPathException
        Parse a string representing an expression. This will accept an XPath expression if called on an ExpressionParser, or an XQuery expression if called on a QueryParser.
        Parameters:
        expression - the expression expressed as a String
        start - offset within the string where parsing is to start
        terminator - character to treat as terminating the expression
        lineNumber - location of the start of the expression, for diagnostics
        env - the static context for the expression
        Returns:
        an Expression object representing the result of parsing
        Throws:
        XPathException - if the expression contains a syntax error
      • customizeTokenizer

        protected void customizeTokenizer​(Tokenizer t)
        Callback to tailor the tokenizer
        Parameters:
        t - the Tokenizer to be customized
      • parseSequenceType

        public SequenceType parseSequenceType​(java.lang.String input,
                                              StaticContext env)
                                       throws XPathException
        Parse a string representing a sequence type
        Parameters:
        input - the string, which should conform to the XPath SequenceType production
        env - the static context
        Returns:
        a SequenceType object representing the type
        Throws:
        XPathException - if any error is encountered
      • parseExpression

        public Expression parseExpression()
                                   throws XPathException
        Parse a top-level Expression: ExprSingle ( ',' ExprSingle )*
        Returns:
        the Expression object that results from parsing
        Throws:
        XPathException - if the expression contains a syntax error
      • parseBinaryExpression

        public Expression parseBinaryExpression​(Expression lhs,
                                                int minPrecedence)
                                         throws XPathException
        Parse a binary expression, using operator precedence parsing. This is used to parse the part of the grammary consisting largely of binary operators distinguished by precedence: from "or expressions" down to "unary expressions". Algorithm for the mainstream binary operators is from Wikipedia article on precedence parsing; operator precedences are from the XQuery specification appendix B.
        Parameters:
        lhs - Left-hand side "basic expression"
        minPrecedence - the minimum precedence of an operator that is to be treated as not terminating the current expression
        Returns:
        the parsed expression
        Throws:
        XPathException - if a static error is found
      • parseTypeswitchExpression

        protected Expression parseTypeswitchExpression()
                                                throws XPathException
        Parse a Typeswitch Expression. This construct is XQuery-only, so the XPath version of this method throws an error unconditionally
        Returns:
        the expression that results from the parsing
        Throws:
        XPathException - if a static error is found
      • parseSwitchExpression

        protected Expression parseSwitchExpression()
                                            throws XPathException
        Parse a Switch Expression. This construct is XQuery-1.1 only, so the XPath and XQuery 1.0 versions of this method throw an error unconditionally
        Returns:
        the expression that results from the parsing
        Throws:
        XPathException - if a static error is found
      • parseValidateExpression

        protected Expression parseValidateExpression()
                                              throws XPathException
        Parse a Validate Expression. This construct is XQuery-only, so the XPath version of this method throws an error unconditionally
        Returns:
        the parsed expression; except that this version of the method always throws an exception
        Throws:
        XPathException - if a static error is found
      • parseExtensionExpression

        protected Expression parseExtensionExpression()
                                               throws XPathException
        Parse an Extension Expression This construct is XQuery-only, so the XPath version of this method throws an error unconditionally
        Returns:
        the parsed expression; except that this version of the method always throws an exception
        Throws:
        XPathException - if a static error is found
      • parseTypePattern

        protected Expression parseTypePattern()
                                       throws XPathException
        Parse a Type Pattern This construct is used only in XSLT patterns, so the XPath version of this method throws an error unconditionally
        Returns:
        the parsed expression; except that this version of the method always throws an exception
        Throws:
        XPathException - if a static error is found
      • parseTryCatchExpression

        protected Expression parseTryCatchExpression()
                                              throws XPathException
        Parse a try/catch Expression This construct is XQuery-1.1 only, so the XPath version of this method throws an error unconditionally
        Returns:
        the parsed expression; except that this version of the method always throws an exception
        Throws:
        XPathException - if a static error is found
      • parseFLWORExpression

        protected Expression parseFLWORExpression()
                                           throws XPathException
        Parse a FOR or LET expression: for $x in expr (',' $y in expr)* 'return' expr let $x := expr (', $y := expr)* 'return' expr This version of the method handles the subset of the FLWOR syntax allowed in XPath
        Returns:
        the resulting subexpression
        Throws:
        XPathException - if any error is encountered
      • parseSequenceType

        public SequenceType parseSequenceType()
                                       throws XPathException
        Parse the sequence type production. The QName must be the name of a built-in schema-defined data type.
        Returns:
        the resulting subexpression
        Throws:
        XPathException - if any error is encountered
      • parseItemType

        public ItemType parseItemType()
                               throws XPathException
        Parse an ItemType within a SequenceType
        Returns:
        the ItemType after parsing
        Throws:
        XPathException - if a static error is found
      • parseFunctionItemType

        protected ItemType parseFunctionItemType()
                                          throws XPathException
        Get the item type used for function items (XPath 3.0 higher order functions)
        Returns:
        the item type representing a function item
        Throws:
        XPathException - if a static error occurs (including the case where XPath 3.0 syntax is not enabled)
      • parseMapItemType

        protected ItemType parseMapItemType()
                                     throws XPathException
        Get the item type used for map items (XPath 3.0)
        Returns:
        the item type of the map
        Throws:
        XPathException - if a parsing error occurs or if the map syntax is not available
      • parseParenthesizedItemType

        protected ItemType parseParenthesizedItemType()
                                               throws XPathException
        Parse a parenthesized item type (XPath 3.0)
        Returns:
        the item type
        Throws:
        XPathException - if a syntax error is found
      • atStartOfRelativePath

        protected boolean atStartOfRelativePath()
        Test whether the current token is one that can start a RelativePathExpression
        Returns:
        the resulting subexpression
      • disallowedAtStartOfRelativePath

        protected boolean disallowedAtStartOfRelativePath()
        Test whether the current token is one that is disallowed after a "leading lone slash". These composite tokens have been parsed as operators, but are not allowed after "/" under the rules of erratum E24
        Returns:
        the resulting subexpression
      • parsePathExpression

        protected Expression parsePathExpression()
                                          throws XPathException
        Parse a PathExpresssion. This includes "true" path expressions such as A/B/C, and also constructs that may start a path expression such as a variable reference $name or a parenthesed expression (A|B). Numeric and string literals also come under this heading.
        Returns:
        the resulting subexpression
        Throws:
        XPathException - if any error is encountered
      • parseRelativePath

        protected Expression parseRelativePath()
                                        throws XPathException
        Parse a relative path (a sequence of steps). Called when the current token immediately follows a separator (/ or //), or an implicit separator (XYZ is equivalent to ./XYZ)
        Returns:
        the resulting subexpression
        Throws:
        XPathException - if any error is encountered
      • parseRemainingPath

        protected Expression parseRemainingPath​(Expression start)
                                         throws XPathException
        Parse the remaining steps of an absolute path expression (one starting in "/" or "//"). Note that the token immediately after the "/" or "//" has already been read, and in the case of "/", it has been confirmed that we have a path expression starting with "/" rather than a standalone "/" expression.
        Parameters:
        start - the initial implicit expression: root() in the case of "/", root()/descendant-or-self::node in the case of "//"
        Returns:
        the completed path expression
        Throws:
        XPathException - if a static error is found
      • parseStepExpression

        protected Expression parseStepExpression​(boolean firstInPattern)
                                          throws XPathException
        Parse a step (including an optional sequence of predicates)
        Parameters:
        firstInPattern - true only if we are parsing the first step in a RelativePathPattern in the XSLT Pattern syntax
        Returns:
        the resulting subexpression
        Throws:
        XPathException - if any error is encountered
      • parsePredicate

        protected Expression parsePredicate()
                                     throws XPathException
        Parse the expression within a predicate. A separate method so it can be overridden
        Returns:
        the expression within the predicate
        Throws:
        XPathException - if a static error is found
      • parseBasicStep

        protected Expression parseBasicStep​(boolean firstInPattern)
                                     throws XPathException
        Parse a basic step expression (without the predicates)
        Parameters:
        firstInPattern - true only if we are parsing the first step in a RelativePathPattern in the XSLT Pattern syntax
        Returns:
        the resulting subexpression
        Throws:
        XPathException - if any error is encountered
      • makeStringLiteral

        protected Literal makeStringLiteral​(java.lang.String currentTokenValue)
                                     throws XPathException
        Method to make a string literal from a token identified as a string literal. This is trivial in XPath, but in XQuery the method is overridden to identify pseudo-XML character and entity references. Note that the job of handling doubled string delimiters is done by the tokenizer.
        Parameters:
        currentTokenValue - the token as read (excluding quotation marks)
        Returns:
        The string value of the string literal
        Throws:
        XPathException - if a static error is found
      • parseConstructor

        protected Expression parseConstructor()
                                       throws XPathException
        Parse a node constructor. This is allowed only in XQuery, so the method throws an error for XPath.
        Returns:
        the expression that results from the parsing
        Throws:
        XPathException - if a static error occurs
      • parseDynamicFunctionCall

        protected Expression parseDynamicFunctionCall​(Expression functionItem)
                                               throws XPathException
        Parse a dynamic function call
        Parameters:
        functionItem - the expression that determines the function to be called
        Returns:
        the expression that results from the parsing
        Throws:
        XPathException - if a static error is found
      • parseNodeTest

        protected NodeTest parseNodeTest​(short nodeType)
                                  throws XPathException
        Parse a NodeTest. One of QName, prefix:*, *:suffix, *, text(), node(), comment(), or processing-instruction(literal?), or element(~,~), attribute(~,~), etc.
        Parameters:
        nodeType - the node type being sought if one is specified
        Returns:
        the resulting NodeTest object
        Throws:
        XPathException - if any error is encountered
      • isNamespaceTestAllowed

        protected boolean isNamespaceTestAllowed()
        Ask whether the syntax namespace-node() is allowed in a node kind test.
        Returns:
        false (currently allowed only in XQuery 3.0)
      • parseMapExpression

        protected Expression parseMapExpression()
                                         throws XPathException
        Parse a map expression. Requires XPath/XQuery 3.0 Provisional syntax map { expr := expr (, expr := expr )*} }
        Returns:
        the map expression
        Throws:
        XPathException - if a static error occurs
      • parseFunctionCall

        protected Expression parseFunctionCall()
                                        throws XPathException
        Parse a function call. function-name '(' ( Expression (',' Expression )* )? ')'
        Returns:
        the resulting subexpression
        Throws:
        XPathException - if any error is encountered
      • resolveFunctionName

        protected StructuredQName resolveFunctionName​(java.lang.String fname)
                                               throws XPathException
        Interpret a function name, returning it as a resolved QName
        Parameters:
        fname - the lexical QName used as the function name; or an EQName presented by the tokenizer as a name in Clark notation
        Returns:
        the Structured QName obtained by resolving any prefix in the function name
        Throws:
        XPathException - if the supplied name is not a valid QName or if its prefix is not in scope
      • parseFunctionArgument

        public Expression parseFunctionArgument()
                                         throws XPathException
        Parse an argument to a function call. Separate method so it can be overridden. With higher-order-function syntax in XPath 3.0/XQuery 3.0, this returns null if the pseudo-argument "?" is found.
        Returns:
        the Expression used as the argument, or null if the argument is the place-holder "?"
        Throws:
        XPathException - if the argument expression does not parse correctly
      • parseLiteralFunctionItem

        protected Expression parseLiteralFunctionItem()
                                               throws XPathException
        Parse a literal function item (introduced in XQuery 1.1) Syntax: QName # integer The QName and # have already been read
        Returns:
        an ExternalObject representing the function item
        Throws:
        XPathException - if a static error is encountered
      • parseInlineFunction

        protected Expression parseInlineFunction()
                                          throws XPathException
        Parse an inline function "function" "(" ParamList? ")" ("as" SequenceType)? EnclosedExpr On entry, "function (" has already been read
        Returns:
        the parsed inline function
        Throws:
        XPathException - if a syntax error is found
      • makeCurriedFunction

        protected Expression makeCurriedFunction​(int offset,
                                                 StructuredQName name,
                                                 Expression[] args,
                                                 IntSet placeMarkers)
                                          throws XPathException
        Process a function call in which one or more of the argument positions are represented as "?" placemarkers (indicating partial application or currying)
        Parameters:
        offset - the position of the expression in the source text
        name - the function name (as if there were no currying)
        args - the arguments (with EmptySequence in the placemarker positions)
        placeMarkers - the positions of the placemarkers @return the curried function
        Returns:
        the curried function
        Throws:
        XPathException - if a static error is found
      • getRangeVariables

        public java.util.Stack<Binding> getRangeVariables()
        Get the stack of in-scope range variables
        Returns:
        the stack of variables
      • setRangeVariables

        public void setRangeVariables​(java.util.Stack<Binding> variables)
        Set a new stack of in-scope range variables
        Parameters:
        variables - the stack of variables
      • declareRangeVariable

        public void declareRangeVariable​(Binding declaration)
                                  throws XPathException
        Declare a range variable (record its existence within the parser). A range variable is a variable declared within an expression, as distinct from a variable declared in the context.
        Parameters:
        declaration - the variable declaration to be added to the stack
        Throws:
        XPathException - if any error is encountered
      • undeclareRangeVariable

        public void undeclareRangeVariable()
        Note when the most recently declared range variable has gone out of scope
      • findRangeVariable

        protected Binding findRangeVariable​(StructuredQName qName)
        Locate a range variable with a given name. (By "range variable", we mean a variable declared within the expression where it is used.)
        Parameters:
        qName - identifies the name of the range variable
        Returns:
        null if not found (this means the variable is probably a context variable); otherwise the relevant RangeVariable
      • setRangeVariableStack

        public void setRangeVariableStack​(java.util.Stack<Binding> stack)
        Set the range variable stack. Used when parsing a nested subexpression inside an attribute constructor.
        Parameters:
        stack - the stack to be used for local variables declared within the expression
      • makeNameCode

        public final int makeNameCode​(java.lang.String qname,
                                      boolean useDefault)
                               throws XPathException
        Make a NameCode, using the static context for namespace resolution
        Parameters:
        qname - The name as written, in the form "[prefix:]localname"; alternatively, a QName in Clark notation ({uri}local)
        useDefault - Defines the action when there is no prefix. If true, use the default namespace URI for element names. If false, use no namespace URI (as for attribute names).
        Returns:
        the namecode, which can be used to identify this name in the name pool
        Throws:
        XPathException - if the name is invalid, or the prefix undeclared
      • makeNameCodeSilently

        public final int makeNameCodeSilently​(java.lang.String qname,
                                              boolean useDefault)
                                       throws XPathException,
                                              QNameException
        Make a NameCode, using the static context for namespace resolution. This variant of the method does not call "grumble" to report any errors to the ErrorListener, it only reports errors by throwing exceptions. This allows the caller to control the message output.
        Parameters:
        qname - The name as written, in the form "[prefix:]localname"
        useDefault - Defines the action when there is no prefix. If true, use the default namespace URI for element names. If false, use no namespace URI (as for attribute names).
        Returns:
        the namecode, which can be used to identify this name in the name pool
        Throws:
        XPathException - if the name is invalid, or the prefix undeclared
        QNameException - if the name is not a lexically valid QName
      • makeStructuredQName

        public final StructuredQName makeStructuredQName​(java.lang.String qname,
                                                         boolean useDefault)
                                                  throws XPathException
        Make a Structured QName, using the static context for namespace resolution
        Parameters:
        qname - The name as written, in the form "[prefix:]localname"; alternatively, a QName in Clark format ({uri}local)
        useDefault - Defines the action when there is no prefix. If true, use the default namespace URI for element names. If false, use no namespace URI (as for attribute names).
        Returns:
        the QName as an instance of StructuredQName
        Throws:
        XPathException - if the name is invalid, or the prefix undeclared
      • makeNodeName

        public final NodeName makeNodeName​(java.lang.String qname,
                                           boolean useDefault)
                                    throws XPathException,
                                           QNameException
        Make a FingerprintedQName, using the static context for namespace resolution
        Parameters:
        qname - The name as written, in the form "[prefix:]localname"; alternatively, a QName in Clark format ({uri}local)
        useDefault - Defines the action when there is no prefix. If true, use the default namespace URI for element names. If false, use no namespace URI (as for attribute names).
        Returns:
        the fingerprinted QName
        Throws:
        XPathException - if the name is invalid, or the prefix undeclared
        QNameException - if the supplied qname is not a lexically valid QName
      • makeNameTest

        public NameTest makeNameTest​(short nodeType,
                                     java.lang.String qname,
                                     boolean useDefault)
                              throws XPathException
        Make a NameTest, using the static context for namespace resolution
        Parameters:
        nodeType - the type of node required (identified by a constant in class Type)
        qname - the lexical QName of the required node; alternatively, a QName in Clark notation ({uri}local)
        useDefault - true if the default namespace should be used when the QName is unprefixed
        Returns:
        a NameTest, representing a pattern that tests for a node of a given node kind and a given name
        Throws:
        XPathException - if the QName is invalid
      • makeNamespaceTest

        public NamespaceTest makeNamespaceTest​(short nodeType,
                                               java.lang.String prefix)
                                        throws XPathException
        Make a NamespaceTest (name:*)
        Parameters:
        nodeType - integer code identifying the type of node required
        prefix - the namespace prefix
        Returns:
        the NamespaceTest, a pattern that matches all nodes in this namespace
        Throws:
        XPathException - if the namespace prefix is not declared
      • makeLocalNameTest

        public LocalNameTest makeLocalNameTest​(short nodeType,
                                               java.lang.String localName)
                                        throws XPathException
        Make a LocalNameTest (*:name)
        Parameters:
        nodeType - the kind of node to be matched
        localName - the requred local name
        Returns:
        a LocalNameTest, a pattern which matches all nodes of a given local name, regardless of namespace
        Throws:
        XPathException - if the local name is invalid
      • setLocation

        protected void setLocation​(Expression exp)
        Set location information on an expression. At present this consists of a simple line number. Needed mainly for XQuery.
        Parameters:
        exp - the expression whose location information is to be set
      • setLocation

        public void setLocation​(Expression exp,
                                int offset)
        Set location information on an expression. At present only the line number is retained. Needed mainly for XQuery. This version of the method supplies an explicit offset (character position within the expression or query), which the tokenizer can convert to a line number and column number.
        Parameters:
        exp - the expression whose location information is to be set
        offset - the character position within the expression (ignoring newlines)
      • setLocation

        public void setLocation​(Clause clause,
                                int offset)
        Set location information on a clause of a FLWOR expression. At present only the line number is retained. Needed mainly for XQuery. This version of the method supplies an explicit offset (character position within the expression or query), which the tokenizer can convert to a line number and column number.
        Parameters:
        clause - the clause whose location information is to be set
        offset - the character position within the expression (ignoring newlines)
      • makeTracer

        public Expression makeTracer​(int startOffset,
                                     Expression exp,
                                     int construct,
                                     StructuredQName qName)
        If tracing, wrap an expression in a trace instruction
        Parameters:
        startOffset - the position of the expression in the soruce
        exp - the expression to be wrapped
        construct - integer constant identifying the kind of construct
        qName - the name of the construct (if applicable)
        Returns:
        the expression that does the tracing
      • isKeyword

        protected boolean isKeyword​(java.lang.String s)
        Test whether the current token is a given keyword.
        Parameters:
        s - The string to be compared with the current token
        Returns:
        true if they are the same
      • normalizeEQName

        protected java.lang.String normalizeEQName​(java.lang.String s)
                                            throws XPathException
        Normalize an EQName. This is written in the source code in the form "uri":local, but by the time it gets here it has been converted to Clark format {uri}local. This method collapses whitespace within the URI
        Parameters:
        s - the EQName in the form of a Clark name
        Returns:
        the normalized EQName
        Throws:
        XPathException - so that the XQuery implementation in a subclass can do so.
      • setScanOnly

        public void setScanOnly​(boolean scanOnly)
        Set that we are parsing in "scan only"
        Parameters:
        scanOnly - true if parsing is to proceed in scan-only mode. In this mode namespace bindings are not yet known, so no attempt is made to look up namespace prefixes.