Class Scanner

  • All Implemented Interfaces:
    Constants, RuntimeConstants
    Direct Known Subclasses:
    Parser

    public class Scanner
    extends java.lang.Object
    implements Constants
    A Scanner for Java tokens. Errors are reported to the environment object.

    The scanner keeps track of the current token, the value of the current token (if any), and the start position of the current token.

    The scan() method advances the scanner to the next token in the input.

    The match() method is used to quickly match opening brackets (ie: '(', '{', or '[') with their closing counter part. This is useful during error recovery.

    An position consists of: ((linenr << WHEREOFFSETBITS) | offset) this means that both the line number and the exact offset into the file are encoded in each position value.

    The compiler treats either "\n", "\r" or "\r\n" as the end of a line.

    WARNING: The contents of this source file are not part of any supported API. Code that depends on them does so at its own risk: they are subject to change or removal without notice.

    • Field Detail

      • OFFSETINC

        public static final long OFFSETINC
        The increment for each character.
        See Also:
        Constant Field Values
      • env

        public Environment env
        Where errors are reported
      • scanComments

        public boolean scanComments
        If true, present all comments as tokens. Contents are not saved, but positions are recorded accurately, so the comment can be recovered from the text. Line terminations are also returned as comment tokens, and may be distinguished by their start and end positions, which are equal (meaning, these tokens contain no chars).
      • token

        public int token
        Current token
      • pos

        public long pos
        The position of the current token
      • prevPos

        public long prevPos
        The position of the previous token
      • ch

        protected int ch
        The current character
      • charValue

        public char charValue
      • intValue

        public int intValue
      • longValue

        public long longValue
      • floatValue

        public float floatValue
      • doubleValue

        public double doubleValue
      • stringValue

        public java.lang.String stringValue
      • radix

        public int radix
      • docComment

        public java.lang.String docComment
      • count

        private int count
      • buffer

        private char[] buffer
    • Constructor Detail

      • Scanner

        public Scanner​(Environment env,
                       java.io.InputStream in)
                throws java.io.IOException
        Create a scanner to scan an input stream.
        Throws:
        java.io.IOException
      • Scanner

        protected Scanner​(Environment env)
        Create a scanner to scan an input stream.
    • Method Detail

      • growBuffer

        private void growBuffer()
      • putc

        private void putc​(int ch)
      • bufferString

        private java.lang.String bufferString()
      • useInputStream

        protected void useInputStream​(java.io.InputStream in)
                               throws java.io.IOException
        Setup input from the given input stream, and scan the first token from it.
        Throws:
        java.io.IOException
      • defineKeyword

        private static void defineKeyword​(int val)
        Define a keyword.
      • skipComment

        private void skipComment()
                          throws java.io.IOException
        Scan a comment. This method should be called once the initial /, * and the next character have been read.
        Throws:
        java.io.IOException
      • scanDocComment

        private java.lang.String scanDocComment()
                                         throws java.io.IOException
        Scan a doc comment. This method should be called once the initial /, * and * have been read. It gathers the content of the comment (witout leading spaces and '*'s) in the string buffer.
        Throws:
        java.io.IOException
      • scanNumber

        private void scanNumber()
                         throws java.io.IOException
        Scan a number. The first digit of the number should be the current character. We may be scanning hex, decimal, or octal at this point
        Throws:
        java.io.IOException
      • scanReal

        private void scanReal()
                       throws java.io.IOException
        Scan a float. We are either looking at the decimal, or we have already seen it and put it into the buffer. We haven't seen an exponent. Scan a float. Should be called with the current character is either the 'e', 'E' or '.'
        Throws:
        java.io.IOException
      • looksLikeZero

        private static boolean looksLikeZero​(java.lang.String token)
      • scanEscapeChar

        private int scanEscapeChar()
                            throws java.io.IOException
        Scan an escape character.
        Returns:
        the character or -1 if it escaped an end-of-line.
        Throws:
        java.io.IOException
      • scanString

        private void scanString()
                         throws java.io.IOException
        Scan a string. The current character should be the opening " of the string.
        Throws:
        java.io.IOException
      • scanCharacter

        private void scanCharacter()
                            throws java.io.IOException
        Scan a character. The current character should be the opening ' of the character constant.
        Throws:
        java.io.IOException
      • scanIdentifier

        private void scanIdentifier()
                             throws java.io.IOException
        Scan an Identifier. The current character should be the first character of the identifier.
        Throws:
        java.io.IOException
      • getEndPos

        public long getEndPos()
        The ending position of the current token
      • getIdToken

        public IdentifierToken getIdToken()
        If the current token is IDENT, return the identifier occurrence. It will be freshly allocated.
      • scan

        public long scan()
                  throws java.io.IOException
        Scan the next token.
        Returns:
        the position of the previous token.
        Throws:
        java.io.IOException
      • xscan

        protected long xscan()
                      throws java.io.IOException
        Throws:
        java.io.IOException
      • match

        public void match​(int open,
                          int close)
                   throws java.io.IOException
        Scan to a matching '}', ']' or ')'. The current token must be a '{', '[' or '(';
        Throws:
        java.io.IOException