Class ExpressionParser

java.lang.Object
net.sf.saxon.expr.parser.ExpressionParser
Direct Known Subclasses:
PatternParser, QueryParser

public class ExpressionParser extends Object
Parser for XPath expressions and XSLT patterns.

This code was originally inspired by James Clark's xt but has been totally rewritten (several times)

The base class handles parsing of XPath 2.0 and XPath 3.0 syntax (switched by a languageVersion variable). Subclasses refine this to handle XQuery syntax (1.0 and 3.0) and XQuery Update syntax.

Author:
Michael Kay
  • Field Details

  • Constructor Details

    • ExpressionParser

      public ExpressionParser()
      Create an expression parser
  • Method Details

    • setCodeInjector

      public void setCodeInjector(CodeInjector injector)
      Set a CodeInjector which can be used to modify or wrap expressions on the tree as the expression is parsed and the tree is constructed. This is typically used to add tracing code.
      Parameters:
      injector - the code injector to be used
    • getCodeInjector

      public CodeInjector getCodeInjector()
      Set a CodeInjector which can be used to modify or wrap expressions on the tree as the expression is parsed and the tree is constructed. This is typically used to add tracing code.
      Returns:
      the code injector in use, if any; or null otherwise
    • getTokenizer

      public Tokenizer getTokenizer()
      Get the tokenizer (the lexical analyzer)
      Returns:
      the tokenizer (the lexical analyzer)
    • getStaticContext

      public StaticContext getStaticContext()
      Get the static context used by this expression parser
      Returns:
      the static context
    • setDefaultContainer

      public void setDefaultContainer(Container container)
      Set the default container for newly constructed expressions
      Parameters:
      container - the default container
    • getDefaultContainer

      public Container getDefaultContainer()
      Get the default container for newly constructed expressions
      Returns:
      the default container
    • getNameChecker

      public NameChecker getNameChecker()
      Get the name checker used by this parser
      Returns:
      the name checker (specific to XML 1.0o
    • nextToken

      public void nextToken() throws XPathException
      Read the next token, catching any exception thrown by the tokenizer
      Throws:
      XPathException - if an invalid token is found
    • expect

      public void expect(int token) throws XPathException
      Expect a given token; fail if the current token is different. Note that this method does not read any tokens.
      Parameters:
      token - the expected token
      Throws:
      XPathException - if the current token is not the expected token
    • grumble

      public void grumble(String message) throws XPathException
      Report a syntax error (a static error with error code XP0003)
      Parameters:
      message - the error message
      Throws:
      XPathException - always thrown: an exception containing the supplied message
    • grumble

      public void grumble(String message, String errorCode) throws XPathException
      Report a static error
      Parameters:
      message - the error message
      errorCode - the error code
      Throws:
      XPathException - always thrown: an exception containing the supplied message
    • grumble

      public void grumble(String message, String errorCode, int offset) throws XPathException
      Report a static error, with location information
      Parameters:
      message - the error message
      errorCode - the error code
      offset - the coded location of the error, or -1 if the location of the current token should be used
      Throws:
      XPathException - always thrown: an exception containing the supplied message
    • grumble

      public void grumble(XPathException error)
    • grumble

      protected void grumble(String message, StructuredQName errorCode, int offset) throws XPathException
      Report a static error
      Parameters:
      message - the error message
      errorCode - the error code
      offset - the coded location of the error, or -1 if the location of the current token should be used
      Throws:
      XPathException - always thrown: an exception containing the supplied message
    • warning

      protected void warning(String message) throws XPathException
      Output a warning message
      Parameters:
      message - the text of the message
      Throws:
      XPathException - if the message cannot be output
    • setLanguage

      public void setLanguage(int language, DecimalValue version)
      Set the current language (XPath or XQuery, XSLT Pattern, or SequenceType)
      Parameters:
      language - one of the constants XPATH, XQUERY, XSLT_PATTERN, SEQUENCE_TYPE
      version - The XPath or XQuery language version. For XQuery the value must be "1.0" or "3.0; for XPath it must be "2.0" or "3.0". Currently support for XQuery 3.0 and XPath 3.0 is incomplete: check the release notes.
    • getLanguage

      protected String getLanguage()
      Get the current language (XPath or XQuery)
      Returns:
      a string representation of the language being parsed, for use in error messages
    • currentTokenDisplay

      protected String currentTokenDisplay()
      Display the current token in an error message
      Returns:
      the display representation of the token
    • parse

      public Expression parse(String expression, int start, int terminator, int lineNumber, StaticContext env) throws XPathException
      Parse a string representing an expression. This will accept an XPath expression if called on an ExpressionParser, or an XQuery expression if called on a QueryParser.
      Parameters:
      expression - the expression expressed as a String
      start - offset within the string where parsing is to start
      terminator - character to treat as terminating the expression
      lineNumber - location of the start of the expression, for diagnostics
      env - the static context for the expression
      Returns:
      an Expression object representing the result of parsing
      Throws:
      XPathException - if the expression contains a syntax error
    • customizeTokenizer

      protected void customizeTokenizer(Tokenizer t)
      Callback to tailor the tokenizer
      Parameters:
      t - the Tokenizer to be customized
    • parseSequenceType

      public SequenceType parseSequenceType(String input, StaticContext env) throws XPathException
      Parse a string representing a sequence type
      Parameters:
      input - the string, which should conform to the XPath SequenceType production
      env - the static context
      Returns:
      a SequenceType object representing the type
      Throws:
      XPathException - if any error is encountered
    • parseExpression

      public Expression parseExpression() throws XPathException
      Parse a top-level Expression: ExprSingle ( ',' ExprSingle )*
      Returns:
      the Expression object that results from parsing
      Throws:
      XPathException - if the expression contains a syntax error
    • parseExprSingle

      public Expression parseExprSingle() throws XPathException
      Parse an ExprSingle
      Returns:
      the resulting subexpression
      Throws:
      XPathException - if any error is encountered
    • parseBinaryExpression

      public Expression parseBinaryExpression(Expression lhs, int minPrecedence) throws XPathException
      Parse a binary expression, using operator precedence parsing. This is used to parse the part of the grammary consisting largely of binary operators distinguished by precedence: from "or expressions" down to "unary expressions". Algorithm for the mainstream binary operators is from Wikipedia article on precedence parsing; operator precedences are from the XQuery specification appendix B.
      Parameters:
      lhs - Left-hand side "basic expression"
      minPrecedence - the minimum precedence of an operator that is to be treated as not terminating the current expression
      Returns:
      the parsed expression
      Throws:
      XPathException - if a static error is found
    • parseTypeswitchExpression

      protected Expression parseTypeswitchExpression() throws XPathException
      Parse a Typeswitch Expression. This construct is XQuery-only, so the XPath version of this method throws an error unconditionally
      Returns:
      the expression that results from the parsing
      Throws:
      XPathException - if a static error is found
    • parseSwitchExpression

      protected Expression parseSwitchExpression() throws XPathException
      Parse a Switch Expression. This construct is XQuery-1.1 only, so the XPath and XQuery 1.0 versions of this method throw an error unconditionally
      Returns:
      the expression that results from the parsing
      Throws:
      XPathException - if a static error is found
    • parseValidateExpression

      protected Expression parseValidateExpression() throws XPathException
      Parse a Validate Expression. This construct is XQuery-only, so the XPath version of this method throws an error unconditionally
      Returns:
      the parsed expression; except that this version of the method always throws an exception
      Throws:
      XPathException - if a static error is found
    • parseExtensionExpression

      protected Expression parseExtensionExpression() throws XPathException
      Parse an Extension Expression This construct is XQuery-only, so the XPath version of this method throws an error unconditionally
      Returns:
      the parsed expression; except that this version of the method always throws an exception
      Throws:
      XPathException - if a static error is found
    • parseTypePattern

      protected Expression parseTypePattern() throws XPathException
      Parse a Type Pattern This construct is used only in XSLT patterns, so the XPath version of this method throws an error unconditionally
      Returns:
      the parsed expression; except that this version of the method always throws an exception
      Throws:
      XPathException - if a static error is found
    • parseTryCatchExpression

      protected Expression parseTryCatchExpression() throws XPathException
      Parse a try/catch Expression This construct is XQuery-1.1 only, so the XPath version of this method throws an error unconditionally
      Returns:
      the parsed expression; except that this version of the method always throws an exception
      Throws:
      XPathException - if a static error is found
    • parseFLWORExpression

      protected Expression parseFLWORExpression() throws XPathException
      Parse a FOR or LET expression: for $x in expr (',' $y in expr)* 'return' expr let $x := expr (', $y := expr)* 'return' expr This version of the method handles the subset of the FLWOR syntax allowed in XPath
      Returns:
      the resulting subexpression
      Throws:
      XPathException - if any error is encountered
    • parseSequenceType

      public SequenceType parseSequenceType() throws XPathException
      Parse the sequence type production. The QName must be the name of a built-in schema-defined data type.
      Returns:
      the resulting subexpression
      Throws:
      XPathException - if any error is encountered
    • parseItemType

      public ItemType parseItemType() throws XPathException
      Parse an ItemType within a SequenceType
      Returns:
      the ItemType after parsing
      Throws:
      XPathException - if a static error is found
    • parseFunctionItemType

      protected ItemType parseFunctionItemType() throws XPathException
      Get the item type used for function items (XPath 3.0 higher order functions)
      Returns:
      the item type representing a function item
      Throws:
      XPathException - if a static error occurs (including the case where XPath 3.0 syntax is not enabled)
    • parseMapItemType

      protected ItemType parseMapItemType() throws XPathException
      Get the item type used for map items (XPath 3.0)
      Returns:
      the item type of the map
      Throws:
      XPathException - if a parsing error occurs or if the map syntax is not available
    • parseParenthesizedItemType

      protected ItemType parseParenthesizedItemType() throws XPathException
      Parse a parenthesized item type (XPath 3.0)
      Returns:
      the item type
      Throws:
      XPathException - if a syntax error is found
    • atStartOfRelativePath

      protected boolean atStartOfRelativePath()
      Test whether the current token is one that can start a RelativePathExpression
      Returns:
      the resulting subexpression
    • disallowedAtStartOfRelativePath

      protected boolean disallowedAtStartOfRelativePath()
      Test whether the current token is one that is disallowed after a "leading lone slash". These composite tokens have been parsed as operators, but are not allowed after "/" under the rules of erratum E24
      Returns:
      the resulting subexpression
    • parsePathExpression

      protected Expression parsePathExpression() throws XPathException
      Parse a PathExpresssion. This includes "true" path expressions such as A/B/C, and also constructs that may start a path expression such as a variable reference $name or a parenthesed expression (A|B). Numeric and string literals also come under this heading.
      Returns:
      the resulting subexpression
      Throws:
      XPathException - if any error is encountered
    • parseRelativePath

      protected Expression parseRelativePath() throws XPathException
      Parse a relative path (a sequence of steps). Called when the current token immediately follows a separator (/ or //), or an implicit separator (XYZ is equivalent to ./XYZ)
      Returns:
      the resulting subexpression
      Throws:
      XPathException - if any error is encountered
    • parseRemainingPath

      protected Expression parseRemainingPath(Expression start) throws XPathException
      Parse the remaining steps of an absolute path expression (one starting in "/" or "//"). Note that the token immediately after the "/" or "//" has already been read, and in the case of "/", it has been confirmed that we have a path expression starting with "/" rather than a standalone "/" expression.
      Parameters:
      start - the initial implicit expression: root() in the case of "/", root()/descendant-or-self::node in the case of "//"
      Returns:
      the completed path expression
      Throws:
      XPathException - if a static error is found
    • parseStepExpression

      protected Expression parseStepExpression(boolean firstInPattern) throws XPathException
      Parse a step (including an optional sequence of predicates)
      Parameters:
      firstInPattern - true only if we are parsing the first step in a RelativePathPattern in the XSLT Pattern syntax
      Returns:
      the resulting subexpression
      Throws:
      XPathException - if any error is encountered
    • parsePredicate

      protected Expression parsePredicate() throws XPathException
      Parse the expression within a predicate. A separate method so it can be overridden
      Returns:
      the expression within the predicate
      Throws:
      XPathException - if a static error is found
    • parseBasicStep

      protected Expression parseBasicStep(boolean firstInPattern) throws XPathException
      Parse a basic step expression (without the predicates)
      Parameters:
      firstInPattern - true only if we are parsing the first step in a RelativePathPattern in the XSLT Pattern syntax
      Returns:
      the resulting subexpression
      Throws:
      XPathException - if any error is encountered
    • testPermittedAxis

      protected void testPermittedAxis(byte axis) throws XPathException
      Throws:
      XPathException
    • parseNumericLiteral

      protected Expression parseNumericLiteral() throws XPathException
      Throws:
      XPathException
    • parseStringLiteral

      protected Expression parseStringLiteral() throws XPathException
      Throws:
      XPathException
    • parseVariableReference

      protected Expression parseVariableReference() throws XPathException
      Throws:
      XPathException
    • makeStringLiteral

      protected Literal makeStringLiteral(String currentTokenValue) throws XPathException
      Method to make a string literal from a token identified as a string literal. This is trivial in XPath, but in XQuery the method is overridden to identify pseudo-XML character and entity references. Note that the job of handling doubled string delimiters is done by the tokenizer.
      Parameters:
      currentTokenValue - the token as read (excluding quotation marks)
      Returns:
      The string value of the string literal
      Throws:
      XPathException - if a static error is found
    • parseConstructor

      protected Expression parseConstructor() throws XPathException
      Parse a node constructor. This is allowed only in XQuery, so the method throws an error for XPath.
      Returns:
      the expression that results from the parsing
      Throws:
      XPathException - if a static error occurs
    • parseDynamicFunctionCall

      protected Expression parseDynamicFunctionCall(Expression functionItem) throws XPathException
      Parse a dynamic function call
      Parameters:
      functionItem - the expression that determines the function to be called
      Returns:
      the expression that results from the parsing
      Throws:
      XPathException - if a static error is found
    • parseNodeTest

      protected NodeTest parseNodeTest(short nodeType) throws XPathException
      Parse a NodeTest. One of QName, prefix:*, *:suffix, *, text(), node(), comment(), or processing-instruction(literal?), or element(~,~), attribute(~,~), etc.
      Parameters:
      nodeType - the node type being sought if one is specified
      Returns:
      the resulting NodeTest object
      Throws:
      XPathException - if any error is encountered
    • isNamespaceTestAllowed

      protected boolean isNamespaceTestAllowed()
      Ask whether the syntax namespace-node() is allowed in a node kind test.
      Returns:
      false (currently allowed only in XQuery 3.0)
    • parseMapExpression

      protected Expression parseMapExpression() throws XPathException
      Parse a map expression. Requires XPath/XQuery 3.0 Provisional syntax map { expr := expr (, expr := expr )*} }
      Returns:
      the map expression
      Throws:
      XPathException - if a static error occurs
    • parseFunctionCall

      protected Expression parseFunctionCall() throws XPathException
      Parse a function call. function-name '(' ( Expression (',' Expression )* )? ')'
      Returns:
      the resulting subexpression
      Throws:
      XPathException - if any error is encountered
    • reportMissingFunction

      public Expression reportMissingFunction(int offset, StructuredQName functionName, Expression[] arguments) throws XPathException
      Throws:
      XPathException
    • resolveFunctionName

      protected StructuredQName resolveFunctionName(String fname) throws XPathException
      Interpret a function name, returning it as a resolved QName
      Parameters:
      fname - the lexical QName used as the function name; or an EQName presented by the tokenizer as a name in Clark notation
      Returns:
      the Structured QName obtained by resolving any prefix in the function name
      Throws:
      XPathException - if the supplied name is not a valid QName or if its prefix is not in scope
    • parseFunctionArgument

      public Expression parseFunctionArgument() throws XPathException
      Parse an argument to a function call. Separate method so it can be overridden. With higher-order-function syntax in XPath 3.0/XQuery 3.0, this returns null if the pseudo-argument "?" is found.
      Returns:
      the Expression used as the argument, or null if the argument is the place-holder "?"
      Throws:
      XPathException - if the argument expression does not parse correctly
    • parseLiteralFunctionItem

      protected Expression parseLiteralFunctionItem() throws XPathException
      Parse a literal function item (introduced in XQuery 1.1) Syntax: QName # integer The QName and # have already been read
      Returns:
      an ExternalObject representing the function item
      Throws:
      XPathException - if a static error is encountered
    • parseInlineFunction

      protected Expression parseInlineFunction() throws XPathException
      Parse an inline function "function" "(" ParamList? ")" ("as" SequenceType)? EnclosedExpr On entry, "function (" has already been read
      Returns:
      the parsed inline function
      Throws:
      XPathException - if a syntax error is found
    • makeCurriedFunction

      protected Expression makeCurriedFunction(int offset, StructuredQName name, Expression[] args, IntSet placeMarkers) throws XPathException
      Process a function call in which one or more of the argument positions are represented as "?" placemarkers (indicating partial application or currying)
      Parameters:
      offset - the position of the expression in the source text
      name - the function name (as if there were no currying)
      args - the arguments (with EmptySequence in the placemarker positions)
      placeMarkers - the positions of the placemarkers @return the curried function
      Returns:
      the curried function
      Throws:
      XPathException - if a static error is found
    • getRangeVariables

      public Stack<Binding> getRangeVariables()
      Get the stack of in-scope range variables
      Returns:
      the stack of variables
    • setRangeVariables

      public void setRangeVariables(Stack<Binding> variables)
      Set a new stack of in-scope range variables
      Parameters:
      variables - the stack of variables
    • declareRangeVariable

      public void declareRangeVariable(Binding declaration) throws XPathException
      Declare a range variable (record its existence within the parser). A range variable is a variable declared within an expression, as distinct from a variable declared in the context.
      Parameters:
      declaration - the variable declaration to be added to the stack
      Throws:
      XPathException - if any error is encountered
    • undeclareRangeVariable

      public void undeclareRangeVariable()
      Note when the most recently declared range variable has gone out of scope
    • findRangeVariable

      protected Binding findRangeVariable(StructuredQName qName)
      Locate a range variable with a given name. (By "range variable", we mean a variable declared within the expression where it is used.)
      Parameters:
      qName - identifies the name of the range variable
      Returns:
      null if not found (this means the variable is probably a context variable); otherwise the relevant RangeVariable
    • setRangeVariableStack

      public void setRangeVariableStack(Stack<Binding> stack)
      Set the range variable stack. Used when parsing a nested subexpression inside an attribute constructor.
      Parameters:
      stack - the stack to be used for local variables declared within the expression
    • makeNameCode

      public final int makeNameCode(String qname, boolean useDefault) throws XPathException
      Make a NameCode, using the static context for namespace resolution
      Parameters:
      qname - The name as written, in the form "[prefix:]localname"; alternatively, a QName in Clark notation ({uri}local)
      useDefault - Defines the action when there is no prefix. If true, use the default namespace URI for element names. If false, use no namespace URI (as for attribute names).
      Returns:
      the namecode, which can be used to identify this name in the name pool
      Throws:
      XPathException - if the name is invalid, or the prefix undeclared
    • makeNameCodeSilently

      public final int makeNameCodeSilently(String qname, boolean useDefault) throws XPathException, QNameException
      Make a NameCode, using the static context for namespace resolution. This variant of the method does not call "grumble" to report any errors to the ErrorListener, it only reports errors by throwing exceptions. This allows the caller to control the message output.
      Parameters:
      qname - The name as written, in the form "[prefix:]localname"
      useDefault - Defines the action when there is no prefix. If true, use the default namespace URI for element names. If false, use no namespace URI (as for attribute names).
      Returns:
      the namecode, which can be used to identify this name in the name pool
      Throws:
      XPathException - if the name is invalid, or the prefix undeclared
      QNameException - if the name is not a lexically valid QName
    • makeStructuredQName

      public final StructuredQName makeStructuredQName(String qname, boolean useDefault) throws XPathException
      Make a Structured QName, using the static context for namespace resolution
      Parameters:
      qname - The name as written, in the form "[prefix:]localname"; alternatively, a QName in Clark format ({uri}local)
      useDefault - Defines the action when there is no prefix. If true, use the default namespace URI for element names. If false, use no namespace URI (as for attribute names).
      Returns:
      the QName as an instance of StructuredQName
      Throws:
      XPathException - if the name is invalid, or the prefix undeclared
    • makeNodeName

      public final NodeName makeNodeName(String qname, boolean useDefault) throws XPathException, QNameException
      Make a FingerprintedQName, using the static context for namespace resolution
      Parameters:
      qname - The name as written, in the form "[prefix:]localname"; alternatively, a QName in Clark format ({uri}local)
      useDefault - Defines the action when there is no prefix. If true, use the default namespace URI for element names. If false, use no namespace URI (as for attribute names).
      Returns:
      the fingerprinted QName
      Throws:
      XPathException - if the name is invalid, or the prefix undeclared
      QNameException - if the supplied qname is not a lexically valid QName
    • makeNameTest

      public NameTest makeNameTest(short nodeType, String qname, boolean useDefault) throws XPathException
      Make a NameTest, using the static context for namespace resolution
      Parameters:
      nodeType - the type of node required (identified by a constant in class Type)
      qname - the lexical QName of the required node; alternatively, a QName in Clark notation ({uri}local)
      useDefault - true if the default namespace should be used when the QName is unprefixed
      Returns:
      a NameTest, representing a pattern that tests for a node of a given node kind and a given name
      Throws:
      XPathException - if the QName is invalid
    • makeNamespaceTest

      public NamespaceTest makeNamespaceTest(short nodeType, String prefix) throws XPathException
      Make a NamespaceTest (name:*)
      Parameters:
      nodeType - integer code identifying the type of node required
      prefix - the namespace prefix
      Returns:
      the NamespaceTest, a pattern that matches all nodes in this namespace
      Throws:
      XPathException - if the namespace prefix is not declared
    • makeLocalNameTest

      public LocalNameTest makeLocalNameTest(short nodeType, String localName) throws XPathException
      Make a LocalNameTest (*:name)
      Parameters:
      nodeType - the kind of node to be matched
      localName - the requred local name
      Returns:
      a LocalNameTest, a pattern which matches all nodes of a given local name, regardless of namespace
      Throws:
      XPathException - if the local name is invalid
    • setLocation

      protected void setLocation(Expression exp)
      Set location information on an expression. At present this consists of a simple line number. Needed mainly for XQuery.
      Parameters:
      exp - the expression whose location information is to be set
    • setLocation

      public void setLocation(Expression exp, int offset)
      Set location information on an expression. At present only the line number is retained. Needed mainly for XQuery. This version of the method supplies an explicit offset (character position within the expression or query), which the tokenizer can convert to a line number and column number.
      Parameters:
      exp - the expression whose location information is to be set
      offset - the character position within the expression (ignoring newlines)
    • setLocation

      public void setLocation(Clause clause, int offset)
      Set location information on a clause of a FLWOR expression. At present only the line number is retained. Needed mainly for XQuery. This version of the method supplies an explicit offset (character position within the expression or query), which the tokenizer can convert to a line number and column number.
      Parameters:
      clause - the clause whose location information is to be set
      offset - the character position within the expression (ignoring newlines)
    • makeTracer

      public Expression makeTracer(int startOffset, Expression exp, int construct, StructuredQName qName)
      If tracing, wrap an expression in a trace instruction
      Parameters:
      startOffset - the position of the expression in the soruce
      exp - the expression to be wrapped
      construct - integer constant identifying the kind of construct
      qName - the name of the construct (if applicable)
      Returns:
      the expression that does the tracing
    • isKeyword

      protected boolean isKeyword(String s)
      Test whether the current token is a given keyword.
      Parameters:
      s - The string to be compared with the current token
      Returns:
      true if they are the same
    • normalizeEQName

      protected String normalizeEQName(String s) throws XPathException
      Normalize an EQName. This is written in the source code in the form "uri":local, but by the time it gets here it has been converted to Clark format {uri}local. This method collapses whitespace within the URI
      Parameters:
      s - the EQName in the form of a Clark name
      Returns:
      the normalized EQName
      Throws:
      XPathException - so that the XQuery implementation in a subclass can do so.
    • setScanOnly

      public void setScanOnly(boolean scanOnly)
      Set that we are parsing in "scan only"
      Parameters:
      scanOnly - true if parsing is to proceed in scan-only mode. In this mode namespace bindings are not yet known, so no attempt is made to look up namespace prefixes.