extending Rake's test tasks

I have a few pure-JavaScript, client-side tests using PhantomJS. These I'd like to integrate with rake test.

Currently I use this:

namespace :test do

  task :client do
    basedir = Rails.root.join("test", "client")
    sh "cd #{basedir} && phantomjs lib/run-qunit.js index.html"
  end

end

task :test => "test:client"

However, this integration is far from perfect; if one of these tests fails, rake aborts. Also, in contrast to :units, :functionals and :integration, there is no summary of the issues at the end (e.g. "6 tests, 21 assertions, 1 failures, 0 errors").

I could extract that data easily enough, but how do I tell Rake to add it to the total test tally?

You are calling via sh a shell command. Ruby does not know, that it is a test. In addition sh seems to stop, if a failure occurs.

You have to do two things: Catch the error and check the result of your call.

An example:

require 'rake'
$summary = Hash.new(0)

def mytest(name, cmd)
  $summary['test'] += 1
  sh cmd do |ok, res|
    if ok 
      $summary['ok'] += 1
    else
      $summary['failure'] += 1
      puts "#{cmd } failed"
    end
  end
end
namespace :test do
  task :one do |tsk|
     mytest(tsk.name, "dir")
  end
  task :two do |tsk|
     mytest(tsk.name, "undefined_cmd")
  end
  task :summary do
    p $summary
  end
end

task :test => "test:one"
task :test => "test:two"
task :test => "test:summary"

shis called with a block to catch failures. Inside the block, I analyse the result (true for ok, false if the script stops with an error. The result is added to a summary hash.

For your use, you may adapt the code and split the code into two files: All test in one file. And the rake file get a Rake::TestTast.

Your test file may look like this:

gem 'test-unit'
require 'test/unit'

class MyTest < Test::Unit::TestCase
  def test_one
    assert_nothing_raised{
      basedir = Rails.root.join("test", "client")        
      res = system("cd #{basedir} && phantomjs lib/run-qunit.js index.html")
      assert_true(res)
    }

  end
  def test_two
    assert_nothing_raised{
      res = `dir` #Test with windows
      assert_match(/C:/, res) #We are in c: 
    }
  end
end

This works only, if your test finish with a exit code. Perhaps you can use `` instead and get the output of your test for a detailed analyze.