class Autotest

Autotest continuously scans the files in your project for changes and runs the appropriate tests. Test failures are run until they have all passed. Then the full test suite is run to ensure that nothing else was inadvertantly broken.

If you want Autotest to start over from the top, hit ^C once. If you want Autotest to quit, hit ^C twice.

Rails:

The autotest command will automatically discover a Rails directory by looking for config/environment.rb. When Rails is discovered, autotest uses RailsAutotest to perform file mappings and other work. See RailsAutotest for details.

Plugins:

Plugins are available by creating a .autotest file either in your project root or in your home directory. You can then write event handlers in the form of:

Autotest.add_hook hook_name { |autotest| ... }

The available hooks are listed in ALL_HOOKS.

See example_dot_autotest.rb for more details.

If a hook returns a true value, it signals to autotest that the hook was handled and should not continue executing hooks.

Naming:

Autotest uses a simple naming scheme to figure out how to map implementation files to test files following the Test::Unit naming scheme.

Test files must be stored in test/
Test files names must start with test_
Test class names must start with Test
Implementation files must be stored in lib/
Implementation files must match up with a test file named test_.*<impl-name>.rb

Strategy:

Find all files and associate them from impl <-> test.
Run all tests.
Scan for failures.
Detect changes in ANY (ruby?. file, rerun all failures + changed files.
Until 0 defects, goto 3.
When 0 defects, goto 2.

Public Class Methods

add_discovery(&proc) click to toggle source

Add a proc to the collection of discovery procs. See autodiscover.

# File lib/autotest.rb, line 174
def self.add_discovery &proc
  @@discoveries << proc
end

add_hook(name, &block) click to toggle source

Add the supplied block to the available hooks, with the given name.

# File lib/autotest.rb, line 844
def self.add_hook name, &block
  HOOKS[name] << block
end

autodiscover() click to toggle source

Automatically find all potential autotest runner styles by searching your loadpath, vendor/plugins, and rubygems for "autotest/discover.rb". If found, that file is loaded and it should register discovery procs with autotest using add_discovery. That proc should return one or more strings describing the user's current environment. Those styles are then combined to dynamically invoke an autotest plugin to suite your environment. That plugin should define a subclass of Autotest with a corresponding name.

Process:

All autotest/discover.rb files loaded.
Those procs determine your styles (eg ["rails", "rspec"]).
Require file by sorting styles and joining (eg 'autotest/rails_rspec').
Invoke run method on appropriate class (eg Autotest::RailsRspec.run).

Example autotest/discover.rb:

Autotest.add_discovery do
  "rails" if File.exist? 'config/environment.rb'
end

# File lib/autotest.rb, line 203
def self.autodiscover
  require 'rubygems'

  # *sigh*
  #
  # This is needed for rspec's hacky discovery mechanism. For some
  # reason rspec2 added generators that create
  # "autotest/discover.rb" right in the project directory instead of
  # keeping it in the rspec gem and properly deciding that the
  # project is an rspec based project or not. See the url for more
  # details:
  #
  # http://rubyforge.org/tracker/?func=detail&atid=1678&aid=28775&group_id=419
  #
  # For the record, the sane way to do it is the bacon way:
  #
  # "Since version 1.0, there is autotest support. You need to tag
  # your test directories (test/ or spec/) by creating an .bacon
  # file there. Autotest then will find it."
  #
  # I'm submitting a counter-patch to rspec to fix stuff properly,
  # but for now I'm stuck with this because their brokenness is
  # documented in multiple books.
  #
  # I'm removing this code once a sane rspec goes out.

  hacky_discovery = Gem::Specification.any? { |s| s.name =~ %r^rspec/ }
  $: << '.' if hacky_discovery

  Gem.find_files("autotest/discover").each do |f|
    load f
  end

  # call all discovery procs and determine the style to use
  @@discoveries.map{ |proc| proc.call }.flatten.compact.sort.uniq
end

new() click to toggle source

Initialize the instance and then load the user's .autotest file, if any.

# File lib/autotest.rb, line 274
def initialize
  # these two are set directly because they're wrapped with
  # add/remove/clear accessor methods
  @exception_list = []
  @test_mappings = []
  @child = nil

  self.completed_re =
    %r\d+ tests, \d+ assertions, \d+ failures, \d+ errors(, \d+ skips)?/
  self.extra_class_map   = {}
  self.extra_files       = []
  self.failed_results_re = %r^\s+\d+\) (?:Failure|Error):\n(.*?)\((.*?)\)/
  self.files_to_test     = new_hash_of_arrays
  self.find_order        = []
  self.known_files       = nil
  self.libs              = %w[. lib test].join(File::PATH_SEPARATOR)
  self.order             = :random
  self.output            = $stderr
  self.prefix            = nil
  self.sleep             = 1
  self.testlib           = "test/unit"
  specified_directories  = ARGV.reject { |arg| arg.start_with?("-") } # options are not directories
  self.find_directories  = specified_directories.empty? ? ['.'] : specified_directories
  self.unit_diff         = nil
  self.latest_results    = nil

  # file in /lib -> run test in /test
  self.add_mapping(%r^lib\/.*\.rb$/) do |filename, _|
    possible = File.basename(filename).gsub '_', '_?' # ' stupid emacs
    files_matching %r^test/.*#{possible}$%
  end

  # file in /test -> run it
  self.add_mapping(%r^test.*\/test_.*rb$/) do |filename, _|
    filename
  end

  default_configs = [File.expand_path('~/.autotest'), './.autotest']
  configs = options[:rc] || default_configs

  configs.each do |f|
    load f if File.exist? f
  end
end

options() click to toggle source

# File lib/autotest.rb, line 69
def self.options
  @@options ||= {}
end

parse_options(args = ARGV) click to toggle source

# File lib/autotest.rb, line 85
  def self.parse_options args = ARGV
    require 'optparse'
    options = {
      :args => args.dup
    }

    OptionParser.new do |opts|
      opts.banner = "        Continuous testing for your ruby app.

          Autotest automatically tests code that has changed. It
          assumes the code is in lib, and tests are in tests. Autotest
          uses plugins to control what happens. You configure plugins
          with require statements in the .autotest file in your
          project base directory, and a default configuration for all
          your projects in the .autotest file in your home directory.

        Usage:
            autotest [options]
".gsub(%r^        /, '')

      opts.on "-f", "--fast-start", "Do not run full tests at start" do
        options[:no_full_after_start] = true
      end

      opts.on("-c", "--no-full-after-failed",
              "Do not run all tests on red->green") do
        options[:no_full_after_failed] = true
      end

      opts.on "-v", "--verbose", "Be annoyingly verbose (debugs .autotest)." do
        options[:verbose] = true
      end

      opts.on "-q", "--quiet", "Be quiet." do
        options[:quiet] = true
      end

      opts.on("-r", "--rc CONF", String, "Override path to config file") do |o|
        options[:rc] = Array(o)
      end

      opts.on("-s", "--style STYLE", String,
              "Manually specify test style. (default: autodiscover)") do |style|
        options[:style] = Array(style)
      end

      opts.on("-w", "--warnings", "Turn on ruby warnings") do
        $-w = true
      end

      opts.on "-h", "--help", "Show this." do
        puts opts
        exit 1
      end
    end.parse! args

    Autotest.options.merge! options

    options
  end

run() click to toggle source

Initialize and run the system.

# File lib/autotest.rb, line 243
def self.run
  new.run
end

runner() click to toggle source

Calculates the autotest runner to use to run the tests.

Can be overridden with --style, otherwise uses ::autodiscover.

# File lib/autotest.rb, line 152
def self.runner
  style = options[:style] || Autotest.autodiscover
  target = Autotest

  unless style.empty? then
    mod = "autotest/#{style.join "_"}"
    puts "loading #{mod}"
    begin
      require mod
    rescue LoadError
      abort "Autotest style #{mod} doesn't seem to exist. Aborting."
    end
    target = Autotest.const_get(style.map {|s| s.capitalize}.join)
  end

  target
end

Public Instance Methods

add_exception(regexp) click to toggle source

Adds regexp to the list of exceptions for find_file. This must be called before the exceptions are compiled.

# File lib/autotest.rb, line 770
def add_exception regexp
  raise "exceptions already compiled" if defined? @exceptions

  @exception_list << regexp
  nil
end

add_mapping(regexp, prepend = false, &proc) click to toggle source

Adds a file mapping, optionally prepending the mapping to the front of the list if prepend is true. regexp should match a file path in the codebase. proc is passed a matched filename and Regexp.last_match. proc should return an array of tests to run.

For example, if test_helper.rb is modified, rerun all tests:

at.add_mapping(%rtest_helper.rb/) do |f, _|
  at.files_matching(%r^test.*rb$/)
end

# File lib/autotest.rb, line 734
def add_mapping regexp, prepend = false, &proc
  if prepend then
    @test_mappings.unshift [regexp, proc]
  else
    @test_mappings.push [regexp, proc]
  end
  nil
end

add_sigint_handler() click to toggle source

Installs a sigint handler.

# File lib/autotest.rb, line 415
def add_sigint_handler
  trap 'INT' do
    Process.kill "KILL", @child if @child

    if self.interrupted then
      self.wants_to_quit = true
    else
      unless hook :interrupt then
        puts "Interrupt a second time to quit"
        self.interrupted = true
        Kernel.sleep 1.5
      end
      raise Interrupt, nil # let the run loop catch it
    end
  end
end

add_sigquit_handler() click to toggle source

Installs a sigquit handler

# File lib/autotest.rb, line 435
def add_sigquit_handler
  trap 'QUIT' do
    restart
  end
end

all_good() click to toggle source

If there are no files left to test (because they've all passed), then all is good.

# File lib/autotest.rb, line 462
def all_good
  files_to_test.empty?
end

clear_exceptions() click to toggle source

Clears the list of exceptions for find_file. This must be called before the exceptions are compiled.

# File lib/autotest.rb, line 791
def clear_exceptions
  raise "exceptions already compiled" if defined? @exceptions
  @exception_list.clear
  nil
end

clear_mappings() click to toggle source

Clears all file mappings. This is DANGEROUS as it entirely disables autotest. You must add at least one file mapping that does a good job of rerunning appropriate tests.

# File lib/autotest.rb, line 758
def clear_mappings
  @test_mappings.clear
  nil
end

consolidate_failures(failed) click to toggle source

Returns a hash mapping a file name to the known failures for that file.

# File lib/autotest.rb, line 483
def consolidate_failures failed
  filters = new_hash_of_arrays

  class_map = Hash[*self.find_order.grep(%r^test/).map { |f| # TODO: ugly
                     [path_to_classname(f), f]
                   }.flatten]
  class_map.merge! self.extra_class_map

  failed.each do |method, klass|
    if class_map.has_key? klass then
      filters[class_map[klass]] << method
    else
      output.puts "Unable to map class #{klass} to a file"
    end
  end

  filters
end

exceptions() click to toggle source

Return a compiled regexp of exceptions for #find_files or nil if no filtering should take place. This regexp is generated from exception_list.

# File lib/autotest.rb, line 802
def exceptions
  unless defined? @exceptions then
    @exceptions = if @exception_list.empty? then
                    nil
                  else
                    Regexp.union(*@exception_list)
                  end
  end

  @exceptions
end

files_matching(regexp) click to toggle source

Returns all known files in the codebase matching regexp.

# File lib/autotest.rb, line 718
def files_matching regexp
  self.find_order.select { |k| k =~ regexp }
end

find_files() click to toggle source

Find the files to process, ignoring temporary files, source configuration management files, etc., and return a Hash mapping filename to modification time.

# File lib/autotest.rb, line 507
def find_files
  result = {}
  targets = self.find_directories + self.extra_files
  self.find_order.clear

  targets.each do |target|
    order = []
    Find.find target do |f|
      Find.prune if f =~ self.exceptions
      Find.prune if f =~ %r^\.\/tmp/    # temp dir, used by isolate

      next unless File.file? f
      next if f =~ %r(swp|~|rej|orig)$/ # temporary/patch files
      next if f =~ %r(,v)$/             # RCS files
      next if f =~ %r\/\.?#/            # Emacs autosave/cvs merge files

      filename = f.sub(%r^\.\//, '')

      result[filename] = File.stat(filename).mtime rescue next
      order << filename
    end
    self.find_order.push(*order.sort)
  end

  result
end

find_files_to_test(files = find_files) click to toggle source

Find the files which have been modified, update the recorded timestamps, and use this to update the files to test. Returns the latest mtime of the files modified or nil when nothing was modified.

# File lib/autotest.rb, line 540
def find_files_to_test files = find_files
  updated = files.select { |filename, mtime| self.last_mtime < mtime }

  # nothing to update or initially run
  unless updated.empty? || self.last_mtime.to_i == 0 then
    p updated if options[:verbose]

    hook :updated, updated
  end

  updated.map { |f,m| test_files_for f }.flatten.uniq.each do |filename|
    self.files_to_test[filename] # creates key with default value
  end

  if updated.empty? then
    nil
  else
    files.values.max
  end
end

get_to_green() click to toggle source

Keep running the tests after a change, until all pass.

# File lib/autotest.rb, line 354
def get_to_green
  begin
    run_tests
    wait_for_changes unless all_good
  end until all_good
end

handle_results(results) click to toggle source

Check results for failures, set the "bar" to red or green, and if there are failures record this.

# File lib/autotest.rb, line 565
def handle_results results
  results = results.gsub(%r\e\[\d+m/, '') # strip ascii color
  failed = results.scan self.failed_results_re
  completed = results[self.completed_re]

  if completed then
    completed = completed.scan(%r(\d+) (\w+)/).map { |v, k| [k, v.to_i] }

    self.latest_results = Hash[*completed.flatten]
    self.files_to_test  = consolidate_failures failed

    color = self.files_to_test.empty? ? :green : :red
    hook color unless $TESTING
  else
    self.latest_results = nil
  end

  self.tainted = true unless self.files_to_test.empty?
end

hook(name, *args) click to toggle source

Call the event hook named name, passing in optional args depending on the hook itself.

Returns false if no hook handled the event.

Hook Writers!

This executes all registered hooks until one returns truthy. Pay attention to the return value of your block!

# File lib/autotest.rb, line 828
def hook name, *args
  deprecated = {
    # none currently
  }

  if deprecated[name] and not HOOKS[name].empty? then
    warn "hook #{name} has been deprecated, use #{deprecated[name]}"
  end

  HOOKS[name].any? { |plugin| plugin[self, *args] }
end

known_files() click to toggle source

Lazy accessor for the #known_files hash.

# File lib/autotest.rb, line 588
def known_files
  unless @known_files then
    @known_files = Hash[*find_order.map { |f| [f, true] }.flatten]
  end
  @known_files
end

make_test_cmd(files_to_test) click to toggle source

Generate the commands to test the supplied files

# File lib/autotest.rb, line 605
def make_test_cmd files_to_test
  cmds = []
  full, partial = reorder(files_to_test).partition { |k,v| v.empty? }
  diff = self.unit_diff
  diff = " | #{diff}" if diff and diff !~ %r^\|/

  unless full.empty? then
    classes = full.map {|k,v| k}.flatten.uniq
    classes.unshift testlib
    classes = classes.join " "
    cmds << "#{ruby_cmd} -e \"%w[#{classes}].each { |f| require f }\"#{diff}"
  end

  partial.each do |klass, methods|
    regexp = Regexp.union(*methods).source
    cmds << "#{ruby_cmd} #{klass} -n \"/^(#{regexp})$/\"#{diff}"
  end

  cmds.join "#{SEP} "
end

new_hash_of_arrays() click to toggle source

# File lib/autotest.rb, line 626
def new_hash_of_arrays
  Hash.new { |h,k| h[k] = [] }
end

old_run_tests() click to toggle source

Alias for: run_tests

options() click to toggle source

# File lib/autotest.rb, line 73
def options
  self.class.options
end

path_to_classname(s) click to toggle source

Convert a path in a string, s, into a class name, changing underscores to CamelCase, etc.

# File lib/autotest.rb, line 470
def path_to_classname s
  sep = File::SEPARATOR
  f = s.sub(%r^test#{sep}/, '').sub(%r\.rb$/, '').split sep
  f = f.map { |path| path.split(%r_|(\d+)/).map { |seg| seg.capitalize }.join }
  f = f.map { |path| path =~ %r^Test/ ? path : "Test#{path}"  }

  f.join '::'
end

remove_exception(regexp) click to toggle source

Removes regexp to the list of exceptions for find_file. This must be called before the exceptions are compiled.

# File lib/autotest.rb, line 781
def remove_exception regexp
  raise "exceptions already compiled" if defined? @exceptions
  @exception_list.delete regexp
  nil
end

remove_mapping(regexp) click to toggle source

Removed a file mapping matching regexp.

# File lib/autotest.rb, line 746
def remove_mapping regexp
  @test_mappings.delete_if do |k,v|
    k == regexp
  end
  nil
end

reorder(files_to_test) click to toggle source

# File lib/autotest.rb, line 630
def reorder files_to_test
  case self.order
  when :alpha then
    files_to_test.sort_by { |k,v| k }
  when :reverse then
    files_to_test.sort_by { |k,v| k }.reverse
  when :random then
    max = files_to_test.size
    files_to_test.sort_by { |k,v| rand max }
  when :natural then
    (self.find_order & files_to_test.keys).map { |f| [f, files_to_test[f]] }
  else
    raise "unknown order type: #{self.order.inspect}"
  end
end

rerun_all_tests() click to toggle source

Rerun the tests from cold (reset state)

# File lib/autotest.rb, line 649
def rerun_all_tests
  reset
  run_tests

  hook :all_good if all_good
end

reset() click to toggle source

Clear all state information about test failures and whether interrupts will kill autotest.

# File lib/autotest.rb, line 660
def reset
  self.files_to_test.clear
  self.find_order.clear

  self.interrupted   = false
  self.known_files   = nil
  self.last_mtime    = T0
  self.tainted       = false
  self.wants_to_quit = false

  hook :reset
end

restart() click to toggle source

# File lib/autotest.rb, line 441
def restart
  Process.kill "KILL", @child if @child

  cmd = [$0, *options[:args]]

  index = $LOAD_PATH.index RbConfig::CONFIG["sitelibdir"]

  if index then
    extra = $LOAD_PATH[0...index]
    cmd = [Gem.ruby, "-I", extra.join(":")] + cmd
  end

  puts cmd.join(" ") if options[:verbose]

  exec(*cmd)
end

ruby() click to toggle source

Determine and return the path of the ruby executable.

# File lib/autotest.rb, line 676
def ruby
  ruby = ENV['RUBY']
  ruby ||= File.join(RbConfig::CONFIG['bindir'],
                     RbConfig::CONFIG['ruby_install_name'])

  ruby.gsub! File::SEPARATOR, File::ALT_SEPARATOR if File::ALT_SEPARATOR

  return ruby
end

ruby_cmd() click to toggle source

Returns the base of the ruby command.

# File lib/autotest.rb, line 598
def ruby_cmd
  "#{prefix}#{ruby} -I#{libs} -rubygems"
end

run() click to toggle source

Repeatedly run failed tests, then all tests, then wait for changes and carry on until killed.

# File lib/autotest.rb, line 323
def run
  hook :initialize
  hook :post_initialize

  reset
  add_sigint_handler

  self.last_mtime = Time.now if options[:no_full_after_start]

  loop do
    begin # ^c handler
      get_to_green
      if tainted? and not options[:no_full_after_failed] then
        rerun_all_tests
      else
        hook :all_good
      end
      wait_for_changes
    rescue Interrupt
      break if wants_to_quit
      reset
    end
  end
  hook :quit
rescue Exception => err
  hook(:died, err) or raise err
end

run_tests() click to toggle source

# File lib/autotest/preload.rb, line 29
def run_tests
  hook :run_command

  new_mtime = self.find_files_to_test
  return unless new_mtime
  self.last_mtime = new_mtime

  begin
    # TODO: deal with unit_diff and partial test runs later
    original_argv = ARGV.dup
    ARGV.clear

    @child = fork do
      trap "QUIT", "DEFAULT"
      trap "INT", "DEFAULT"
      files_to_test.keys.each do |file|
        load file
      end
    end
    Process.wait
  ensure
    @child = nil
    ARGV.replace original_argv
  end

  hook :ran_command
end

Also aliased as: old_run_tests

test_files_for(filename) click to toggle source

Return the name of the file with the tests for filename by finding a test_mapping that matches the file and executing the mapping's proc.

# File lib/autotest.rb, line 691
def test_files_for filename
  result = @test_mappings.find { |file_re, ignored| filename =~ file_re }

  p :test_file_for => [filename, result.first] if result and $DEBUG

  result = result.nil? ? [] : [result.last.call(filename, $~)].flatten

  output.puts "No tests matched #{filename}" if
    (options[:verbose] or $TESTING) and result.empty?

  result.sort.uniq.select { |f| known_files[f] }
end

wait_for_changes() click to toggle source

Sleep then look for files to test, until there are some.

# File lib/autotest.rb, line 707
def wait_for_changes
  hook :waiting
  Kernel.sleep self.sleep until find_files_to_test
end