Class Tokenizer

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable

    class Tokenizer
    extends java.lang.Object
    implements java.io.Closeable
    • Field Summary

      Fields 
      Modifier and Type Field Description
      (package private) java.lang.StringBuilder buffer  
      (package private) java.io.File file  
      (package private) int lastChar  
      (package private) int lineNumber  
      (package private) java.lang.String lineSeparator  
      (package private) java.io.Reader reader  
      (package private) java.lang.String text  
    • Constructor Summary

      Constructors 
      Constructor Description
      Tokenizer​(java.io.File file)  
      Tokenizer​(java.io.File file, java.lang.String encoding)  
      Tokenizer​(java.io.Reader reader, java.io.File file, int lineNumber)  
      Tokenizer​(java.lang.String text, java.io.File file, int lineNumber)  
    • Field Detail

      • file

        java.io.File file
      • text

        java.lang.String text
      • reader

        java.io.Reader reader
      • lineSeparator

        java.lang.String lineSeparator
      • lastChar

        int lastChar
      • lineNumber

        int lineNumber
      • buffer

        java.lang.StringBuilder buffer
    • Constructor Detail

      • Tokenizer

        Tokenizer​(java.io.Reader reader,
                  java.io.File file,
                  int lineNumber)
      • Tokenizer

        Tokenizer​(java.lang.String text,
                  java.io.File file,
                  int lineNumber)
      • Tokenizer

        Tokenizer​(java.io.File file)
           throws java.io.IOException
        Throws:
        java.io.IOException
      • Tokenizer

        Tokenizer​(java.io.File file,
                  java.lang.String encoding)
           throws java.io.IOException
        Throws:
        java.io.IOException
    • Method Detail

      • filterLines

        public void filterLines​(java.lang.String[] patterns,
                                boolean skip)
                         throws java.io.IOException
        Throws:
        java.io.IOException
      • close

        public void close()
                   throws java.io.IOException
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Throws:
        java.io.IOException
      • readChar

        int readChar()
              throws java.io.IOException
        Throws:
        java.io.IOException
      • nextToken

        public Token nextToken()
                        throws java.io.IOException
        Throws:
        java.io.IOException
      • tokenize

        Token[] tokenize()