Parent

Class/Module Index [+]

Quicksearch

YARD::Parser::SourceParser

Responsible for parsing a source file into the namespace. Parsing also invokes handlers to process the parsed statements and generate any code objects that may be recognized.

Custom Parsers

SourceParser allows custom parsers to be registered and called when a certain filetype is recognized. To register a parser and hook it up to a set of file extensions, call {register_parser_type}

@see register_parser_type @see Handlers::Base @see CodeObjects::Base

Constants

ENCODING_BYTE_ORDER_MARKS

Byte order marks for various encodings @since 0.7.0

ENCODING_LINE
SHEBANG_LINE

Attributes

parser_type[R]

@return [Symbol] the default parser type (defaults to :ruby)

parser_type_extensions[R]

@return [Hash] a list of registered parser type extensions @private @since 0.5.6

parser_types[R]

@return [Hash{Symbol=>Object}] a list of registered parser types @private @since 0.5.6

contents[R]

@return [String] the contents of the file to be parsed @since 0.7.0

file[RW]

@return [String] the filename being parsed by the parser.

globals[R]

@return [OpenStruct] an open struct containing arbitrary global state

shared between files and handlers.

@since 0.7.0

parser_type[R]

@return [Symbol] the parser type associated with the parser instance.

This should be set by the {#initialize constructor}.

Public Class Methods

after_parse_file(&block) click to toggle source

Registers a callback to be called after an individual file is parsed. The block passed to this method will be called on subsequent parse calls.

To register a callback that is called after the entire list of files is processed, see {after_parse_list}.

@example Printing the length of each file after it is parsed

SourceParser.after_parse_file do |parser|
  puts "#{parser.file} is #{parser.contents.size} characters"
end
YARD.parse('lib/**/*.rb')
# prints:
"lib/foo.rb is 1240 characters"
"lib/foo_bar.rb is 248 characters"

@yield [parser] the yielded block is called once after each file

that is parsed. This might happen many times for a single codebase.

@yieldparam [SourceParser] parser the parser object that parsed

the file.

@yieldreturn [void] the return value for the block is ignored. @return [Proc] the yielded block @see before_parse_file @see after_parse_list @since 0.7.0

# File lib/yard/parser/source_parser.rb, line 321
def after_parse_file(&block)
  after_parse_file_callbacks << block
end
after_parse_file_callbacks() click to toggle source

@return [Array<Proc>] the list of callbacks to be called after

parsing a file. Should only be used for testing.

@since 0.7.0

# File lib/yard/parser/source_parser.rb, line 349
def after_parse_file_callbacks
  @after_parse_file_callbacks ||= []
end
after_parse_list(&block) click to toggle source

Registers a callback to be called after a list of files is parsed via {parse}. The block passed to this method will be called on subsequent parse calls.

@example Printing results after parsing occurs

SourceParser.after_parse_list do
  puts "Finished parsing!"
end
YARD.parse
# Prints "Finished parsing!" after parsing files

@yield [files, globals] the yielded block is called once before

parsing all files

@yieldparam [Array<String>] files the list of files that will be parsed. @yieldparam [OpenStruct] globals a global structure to store arbitrary

state for post processing (see {Handlers::Processor#globals})

@yieldreturn [void] the return value for the block is ignored. @return [Proc] the yielded block @see before_parse_list @see before_parse_file @since 0.7.0

# File lib/yard/parser/source_parser.rb, line 255
def after_parse_list(&block)
  after_parse_list_callbacks << block
end
after_parse_list_callbacks() click to toggle source

@return [Array<Proc>] the list of callbacks to be called after

parsing a list of files. Should only be used for testing.

@since 0.7.0

# File lib/yard/parser/source_parser.rb, line 335
def after_parse_list_callbacks
  @after_parse_list_callbacks ||= []
end
before_parse_file(&block) click to toggle source

Registers a callback to be called before an individual file is parsed. The block passed to this method will be called on subsequent parse calls.

To register a callback that is called before the entire list of files is processed, see {before_parse_list}.

@example Installing a simple callback

SourceParser.before_parse_file do |parser|
  puts "I'm parsing #{parser.file}"
end
YARD.parse('lib/**/*.rb')
# prints:
"I'm parsing lib/foo.rb"
"I'm parsing lib/foo_bar.rb"
"I'm parsing lib/last_file.rb"

@example Cancel parsing of any test_*.rb files

SourceParser.before_parse_file do |parser|
  return false if parser.file =~ /^test_.+\.rb$/
end

@yield [parser] the yielded block is called once before each

file that is parsed. This might happen many times for a single
codebase.

@yieldparam [SourceParser] parser the parser object that will {parse}

the file.

@yieldreturn [Boolean] if the block returns false, parsing for

the file is cancelled.

@return [Proc] the yielded block @see after_parse_file @see before_parse_list @since 0.7.0

# File lib/yard/parser/source_parser.rb, line 292
def before_parse_file(&block)
  before_parse_file_callbacks << block
end
before_parse_file_callbacks() click to toggle source

@return [Array<Proc>] the list of callbacks to be called before

parsing a file. Should only be used for testing.

@since 0.7.0

# File lib/yard/parser/source_parser.rb, line 342
def before_parse_file_callbacks
  @before_parse_file_callbacks ||= []
end
before_parse_list(&block) click to toggle source

Registers a callback to be called before a list of files is parsed via {parse}. The block passed to this method will be called on subsequent parse calls.

@example Installing a simple callback

SourceParser.before_parse_list do |files, globals|
  puts "Starting to parse..."
end
YARD.parse('lib/**/*.rb')
# prints "Starting to parse..."

@example Setting global state

SourceParser.before_parse_list do |files, globals|
  globals.method_count = 0
end
SourceParser.after_parse_list do |files, globals|
  puts "Found #{globals.method_count} methods"
end
class MyCountHandler < Handlers::Ruby::Base
  handles :def, :defs
  process { globals.method_count += 1 }
end
YARD.parse
# Prints: "Found 37 methods"

@example Using a global callback to cancel parsing

SourceParser.before_parse_list do |files, globals|
  return false if files.include?('foo.rb')
end

YARD.parse(['foo.rb', 'bar.rb']) # callback cancels this method
YARD.parse('bar.rb') # parses normally

@yield [files, globals] the yielded block is called once before

parsing all files

@yieldparam [Array<String>] files the list of files that will be parsed. @yieldparam [OpenStruct] globals a global structure to store arbitrary

state for post processing (see {Handlers::Processor#globals})

@yieldreturn [Boolean] if the block returns false, parsing is

cancelled.

@return [Proc] the yielded block @see after_parse_list @see before_parse_file @since 0.7.0

# File lib/yard/parser/source_parser.rb, line 231
def before_parse_list(&block)
  before_parse_list_callbacks << block
end
before_parse_list_callbacks() click to toggle source

@return [Array<Proc>] the list of callbacks to be called before

parsing a list of files. Should only be used for testing.

@since 0.7.0

# File lib/yard/parser/source_parser.rb, line 328
def before_parse_list_callbacks
  @before_parse_list_callbacks ||= []
end
new(parser_type = SourceParser.parser_type, globals1 = nil, globals2 = nil) click to toggle source

@overload initialize(parser_type = SourceParser.parser_type, globals = nil)

Creates a new parser object for code parsing with a specific parser type.

@param [Symbol] parser_type the parser type to use
@param [OpenStruct] globals global state to be re-used across separate source files
# File lib/yard/parser/source_parser.rb, line 403
def initialize(parser_type = SourceParser.parser_type, globals1 = nil, globals2 = nil)
  globals = [true, false].include?(globals1) ? globals2 : globals1
  @file = '(stdin)'
  @globals = globals || OpenStruct.new
  self.parser_type = parser_type
end
parse(paths = ["{lib,app}/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level) click to toggle source

Parses a path or set of paths

@param [String, Array<String>] paths a path, glob, or list of paths to

parse

@param [Array<String, Regexp>] excluded a list of excluded path matchers @param [Fixnum] level the logger level to use during parsing. See

{YARD::Logger}

@return [void]

# File lib/yard/parser/source_parser.rb, line 93
def parse(paths = ["{lib,app}/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level)
  log.debug("Parsing #{paths.inspect} with `#{parser_type}` parser")
  excluded = excluded.map do |path|
    case path
    when Regexp; path
    else Regexp.new(path.to_s, Regexp::IGNORECASE)
    end
  end
  files = [paths].flatten.
    map {|p| File.directory?(p) ? "#{p}/**/*.{rb,c}" : p }.
    map {|p| p.include?("*") ? Dir[p].sort_by {|f| f.length } : p }.flatten.
    reject {|p| !File.file?(p) || excluded.any? {|re| p =~ re } }

  log.enter_level(level) do
    parse_in_order(*files.uniq)
  end
end
parse_string(content, ptype = parser_type) click to toggle source

Parses a string content

@param [String] content the block of code to parse @param [Symbol] ptype the parser type to use. See {parser_type}. @return the parser object that was used to parse content

# File lib/yard/parser/source_parser.rb, line 116
def parse_string(content, ptype = parser_type)
  new(ptype).parse(StringIO.new(content))
end
parser_type=(value) click to toggle source
# File lib/yard/parser/source_parser.rb, line 81
def parser_type=(value)
  @parser_type = validated_parser_type(value)
end
parser_type_extensions=(value) click to toggle source
# File lib/yard/parser/source_parser.rb, line 161
def parser_type_extensions=(value) @@parser_type_extensions = value end
parser_type_for_extension(extension) click to toggle source

Finds a parser type that is registered for the extension. If no type is found, the default Ruby type is returned.

@return [Symbol] the parser type to be used for the extension @since 0.5.6

# File lib/yard/parser/source_parser.rb, line 168
def parser_type_for_extension(extension)
  type = parser_type_extensions.find do |t, exts|
    [exts].flatten.any? {|ext| ext === extension }
  end
  validated_parser_type(type ? type.first : :ruby)
end
parser_types=(value) click to toggle source
# File lib/yard/parser/source_parser.rb, line 153
def parser_types=(value) @@parser_types = value end
register_parser_type(type, parser_klass, extensions = nil) click to toggle source

Registers a new parser type.

@example Registering a parser for "java" files

SourceParser.register_parser_type :java, JavaParser, 'java'

@param [Symbol] type a symbolic name for the parser type @param [Base] parser_klass a class that implements parsing and tokenization @param [Array<String>, String, Regexp] extensions a list of extensions or a

regex to match against the file extension

@return [void] @see Parser::Base

# File lib/yard/parser/source_parser.rb, line 139
def register_parser_type(type, parser_klass, extensions = nil)
  unless Base > parser_klass
    raise ArgumentError, "expecting parser_klass to be a subclass of YARD::Parser::Base"
  end
  parser_type_extensions[type.to_sym] = extensions if extensions
  parser_types[type.to_sym] = parser_klass
end
tokenize(content, ptype = parser_type) click to toggle source

Tokenizes but does not parse the block of code

@param [String] content the block of code to tokenize @param [Symbol] ptype the parser type to use. See {parser_type}. @return [Array] a list of tokens

# File lib/yard/parser/source_parser.rb, line 125
def tokenize(content, ptype = parser_type)
  new(ptype).tokenize(content)
end
validated_parser_type(type) click to toggle source

Returns the validated parser type. Basically, enforces that :ruby type is never set if the Ripper library is not available

@param [Symbol] type the parser type to set @return [Symbol] the validated parser type @private

# File lib/yard/parser/source_parser.rb, line 181
def validated_parser_type(type)
  !defined?(::Ripper) && type == :ruby ? :ruby18 : type
end

Public Instance Methods

parse(content = __FILE__) click to toggle source

The main parser method. This should not be called directly. Instead, use the class methods {parse} and {parse_string}.

@param [String, read, Object] content the source file to parse @return [Object, nil] the parser object used to parse the source

# File lib/yard/parser/source_parser.rb, line 415
def parse(content = __FILE__)
  case content
  when String
    @file = File.cleanpath(content)
    content = convert_encoding(File.read_binary(file))
    checksum = Registry.checksum_for(content)
    return if Registry.checksums[file] == checksum

    if Registry.checksums.has_key?(file)
      log.info "File '#{file}' was modified, re-processing..."
    end
    Registry.checksums[@file] = checksum
    self.parser_type = parser_type_for_filename(file)
  else
    content = content.read if content.respond_to? :read
  end

  @contents = content
  @parser = parser_class.new(content, file)

  self.class.before_parse_file_callbacks.each do |cb|
    return @parser if cb.call(self) == false
  end

  @parser.parse
  post_process

  self.class.after_parse_file_callbacks.each do |cb|
    cb.call(self)
  end

  @parser
rescue ArgumentError, NotImplementedError => e
  log.warn("Cannot parse `#{file}': #{e.message}")
  log.backtrace(e, :warn)
rescue ParserSyntaxError => e
  log.warn(e.message.capitalize)
  log.backtrace(e, :warn)
end
tokenize(content) click to toggle source

Tokenizes but does not parse the block of code using the current {parser_type}

@param [String] content the block of code to tokenize @return [Array] a list of tokens

# File lib/yard/parser/source_parser.rb, line 459
def tokenize(content)
  @parser = parser_class.new(content, file)
  @parser.tokenize
end

[Validate]

Generated with the Darkfish Rdoc Generator 2.