This article explains all steps you need to perform to setup your Continuous Integration System with TeamCity + PhantomJS + CoffeeScript (optionally) + Jasmine and/or CasperJS :)

There's an increasing demand for interactive web applications which dynamically load data using AJAX and handle user interaction with lots of fancy JavaScript stuff. That's why we have significant more Javascript code in our web apps than it was 10 years ago (you still remember how "hip" it was to use JavaScript back in 2000 as a web developer?).

Javascript code is essential for modern web applications. It needs to be tested. It wants to be tested. We needed a setup which allows to run our CasperJS and Jasmine test suites both locally and on our build server (TeamCity). This article covers the setup of TeamCity for local testing and especially automated headless testing of Javascript code in detail.

Most of our Javascript tests are written in CoffeeScript. CoffeeScript provides syntactic sugar to write cleaner, more readable code which transcompiles to JavaScript. However using CoffeeScript makes the setup of the buildserver slightly more sophisticated. Not too hard though. Give it a chance! ;)

Jasmine

A behaviour-driven JS testframework we use for writing JavaScript unit tests. It easily integrates with Ruby projects. Syntactically it looks quite similar to RSpec:

describe("A suite", function() {
	it("should work flawlessly", function() {
		expect(true).toBe(true);
	});
});

describe 'A suite', ->
	it 'should work flawlessly', ->
		expect(true).toBe true

Links:

• http://coffeescriptcookbook.com/chapters/testing/testing_with_jasmine
• http://railscasts.com/episodes/261-testing-javascript-with-jasmine-revised
• http://pivotal.github.com/jasmine/

After installing the gem jasminerice in your rails app, it's really simple to execute jasmine tests locally: Simply start up your rails server and fire up http://localhost:3000/jasmine in the browser.

CasperJS

CasperJS is a navigation scripting and testing utility used by us to navigate through our web apps, create screenshots, fill and submit forms and much more. To get started simply add the gem casperjs to your rails project.

  test_server_pid_file = "tmp/pids/test_server.pid"
  app_port             = 8787
  desc "Start Testing Server"
  task :start_test_server do
    counter = 0

    Thread.new {
      command_line = "bundle exec rails server -p #{app_port} -e test -P #{test_server_pid_file}"
      puts "Starting: #{command_line}"
      system command_line
    }

    while (not File.exist?(test_server_pid_file)) && counter < 90 do
      counter += 1
      sleep 2
    end
    if counter >= 30
      STDERR.puts "Start took too long!"
    else
      puts "Test server running ..."
    end
  end

  desc "Stop Testing Server"
  task :stop_test_server do

    puts "Stopping test server ..."
    pid = IO.read(test_server_pid_file).to_i rescue nil

    if pid.present?
      system("kill -9 #{pid}")
      FileUtils.rm(test_server_pid_file)
      puts "... Test server stopped"
    else
      STDERR.puts "Cannot stop server - no pid file found!"
    end
  end


  desc "run Casper JS Tests, starts rails server,run the tests and then stop the server "
  task :run_casper_js_tests => [:init, "db:fixtures:load", "projectname:start_test_server" ] do
    begin
      Rake::Task["projectname:start_test_server"].invoke

      spec_path = Rails.root.join("spec/javascripts/casperjs/")
      log_path  = Rails.root.join("log")
      puts "Running CasperJS Specs in #{spec_path}"
      # 1. first runs pre.js
      # 2. then runs all JS and CoffeeScript Specs in directory spec_path
      # 3. then runs post.js
      # 4. finally stores result to log/casper_spec_output.xml
      system("casperjs test \
              --testhost=http://localhost:#{app_port} \
              --save_to_xml=#{File.join(log_path, 'casper_spec_output.xml')} \
              --pre=#{File.join(spec_path, 'pre.js.coffee')} \
              --post=#{File.join(spec_path, 'post.js.coffee')} \
              --log-level=info \
              #{File.join(spec_path)}")
    ensure
      Rake::Task["projectname:stop_test_server"].invoke
    end
  end

You can then run your CasperJS tests (let it be JS- or coffeescript-tests) by running the new Raketask.

TeamCity Setup

TeamCity is a popular Continuous Integration Server developed by JetBrains. We use it to automatically build and test our Java, Rails and iOS projects. This chapter describes how to setup TeamCity for automatic JavaScript testing of Rails projects.

We use PhantomJS - the scriptable headless WebKit engine - to execute our JS tests on the build server. Therefore initially we need to install a recent version of phantomjs on all buildagents which shall run js code. Here's an example for PhantomJS v.1.8.1:

ssh buildagent_1
cd /usr/local/share
wget http://phantomjs.googlecode.com/files/phantomjs-1.8.1-linux-i686.tar.bz2
tar xjf phantomjs-1.8.1-linux-i686.tar.bz2
sudo ln -s /usr/local/share/phantomjs-1.8.1-linux-i686/bin/phantomjs /usr/bin/phantomjs 
rm phantomjs-1.8.1-linux-i686.tar.bz2

For the content of the following files we got heavy inspiration from Dan's great article http://blog.danmerino.com/continuos-integration-ci-for-javascript-jasmine-and-teamcity/.
Add the following files to your project:

• /script/teamcity/test_runner.js: PhantomJS script which triggers the headless execution of the JS tests

  console.log('PhantomJS started TestRunner which loads a web page with a TeamCity Reporter...');
  var page = new WebPage();
  var system = require('system');
  var checkout_dir = system.args[1]; // pass checkout_dir parameter
  
  //Open local teamcity_reporter.html
  var url = "file://localhost" + checkout_dir + "/script/teamcity/teamcity_reporter.html";
  phantom.viewportSize = {width: 800, height: 600};
  //Required because PhantomJS sandboxes the website and doesn't show up the console messages form that page by default
  page.onConsoleMessage = function (msg) {
      console.log(msg);   // Pass all page logs to stdout
  
      if (msg && msg.indexOf("##jasmine.reportRunnerResults") !== -1) {
          phantom.exit();
      }
  };
  //Open the website with the teamcity reporter
  page.open(url, function (status) {
      //Page is loaded!
      if (status !== 'success') {
          console.log('Unable to load the address!');
      } else {
          //Using a delay to make sure the JavaScript is executed in the browser
          window.setTimeout(function () {
              page.render("output.png");
              phantom.exit();
          }, 200);
      }
  });
  

• /script/teamcity/teamcity_reporter.html: HTML page which is being rendered in the PhantomJS browser. Tells Jasmine which tests it shall execute and registers TeamCity reporter with Jasmine

• /script/teamcity/jasmine.teamcity_reporter.js: hands over spec results to TeamCity build server

  (function() {
      if (! jasmine) {
          throw new Exception("jasmine library does not exist in global namespace!");
      }
  
      /**
       * Basic reporter that outputs spec results to for the Teamcity build system
       *
       * Usage:
       *
       * jasmine.getEnv().addReporter(new jasmine.TeamcityReporter());
       * jasmine.getEnv().execute();
       */
      var TeamcityReporter = function() {
          this.started = false;
          this.finished = false;
      };
  
      TeamcityReporter.prototype = {
          reportRunnerResults: function(runner) { },
  
          reportRunnerStarting: function(runner) { },
  
          reportSpecResults: function(spec) { },
  
          reportSpecStarting: function(spec) { },
  
          reportSuiteResults: function(suite) {
              var results = suite.results();
              var path = [];
              while(suite) {
                  path.unshift(suite.description);
                  suite = suite.parentSuite;
              }
              var description = path.join(' ');
  
              this.log("##teamcity[testSuiteStarted name='" + this.escapeTeamcityString(description) + "']");
  
              var outerThis = this;
              var eachSpecFn = function(spec){
                  if (spec.description) {
                      outerThis.log("##teamcity[testStarted name='" + outerThis.escapeTeamcityString(spec.description) + "' captureStandardOutput='true']");
                      var specResultFn = function(result){
                          if (!result.passed_) {
                              outerThis.log("##teamcity[testFailed name='" + outerThis.escapeTeamcityString(spec.description) + "' message='|[FAILED|]' details='" + outerThis.escapeTeamcityString(result.trace.stack) + "']");
                          }
                      };
  
                      for (var j = 0, jlen = spec.items_.length; j < jlen; j++) {
                          specResultFn(spec.items_[j]);
                      }
                      outerThis.log("##teamcity[testFinished name='" + outerThis.escapeTeamcityString(spec.description) + "']");
                  }
              };
              for (var i = 0, ilen = results.items_.length; i < ilen; i++) {
                  eachSpecFn(results.items_[i]);
              }
  
              this.log("##teamcity[testSuiteFinished name='" + outerThis.escapeTeamcityString(description) + "']");
          },
  
          log: function(str) {
              var console = jasmine.getGlobal().console;
              if (console && console.log) {
                  console.log(str);
              }
          },
  
          hasGroupedConsole: function() {
              var console = jasmine.getGlobal().console;
              return console && console.info && console.warn && console.group && console.groupEnd && console.groupCollapsed;
          },
  
          escapeTeamcityString: function(message) {
              if(!message) {
                  return "";
              }
  
              return message.replace(/\|/g, "||")
                  .replace(/\'/g, "|'")
                  .replace(/\n/g, "|n")
                  .replace(/\r/g, "|r")
                  .replace(/\u0085/g, "|x")
                  .replace(/\u2028/g, "|l")
                  .replace(/\u2029/g, "|p")
                  .replace(/\[/g, "|[")
                  .replace(/]/g, "|]");
          }
      };
  
      function suiteResults(suite) {
          console.group(suite.description);
          var specs = suite.specs();
          for (var i in specs) {
              if (specs.hasOwnProperty(i)) {
                  specResults(specs[i]);
              }
          }
          var suites = suite.suites();
          for (var j in suites) {
              if (suites.hasOwnProperty(j)) {
                  suiteResults(suites[j]);
              }
          }
          console.groupEnd();
      }
  
      function specResults(spec) {
          var results = spec.results();
          if (results.passed() && console.groupCollapsed) {
              console.groupCollapsed(spec.description);
          } else {
              console.group(spec.description);
          }
          var items = results.getItems();
          for (var k in items) {
              if (items.hasOwnProperty(k)) {
                  itemResults(items[k]);
              }
          }
          console.groupEnd();
      }
  
      function itemResults(item) {
          if (item.passed && !item.passed()) {
              console.warn({actual:item.actual,expected: item.expected});
              item.trace.message = item.matcherName;
              console.error(item.trace);
          } else {
              console.info('Passed');
          }
      }
  
      // export public
      jasmine.TeamcityReporter = TeamcityReporter;
  })();
  

• /spec/javascripts/jasmine_spec_suite.js: Lists the jasmine specs which shall be tested by TeamCity

  // This is a manifest file that'll be compiled into application.js, which will include all the
  // files listed below.
  // Any JavaScript/Coffee file within this directory, lib/assets/javascripts,
  // vendor/assets/javascripts, or vendor/assets/javascripts of plugins,
  // if any, can be referenced here using a relative path.
  // It's not advisable to add code directly here, but if you do, it'll appear at the bottom of the
  // the compiled file.
  //
  // WARNING: THE FIRST BLANK LINE MARKS THE END OF WHAT'S TO BE
  // PROCESSED, ANY BLANK LINE SHOULD GO AFTER THE REQUIRES BELOW.
  //
  // add FACTORIES
  //= require ./factories/dog_factory
  // add SPECS
  //= require fat_dog_spec
  //= require_self
  

• /Rakefile: task which compiles coffeescript files to one big js file

    desc "Asset Precompile for Coffeescript Jasmine Tests"
    task :jasmine_asset_precompile => ["assets:precompile"] do
      # compile JS specs after creating assets
      compile_jasmine_coffeescripts
    end
  
    # compile all JS specs to one big js file in /spec/javascript/assets dir
    def compile_jasmine_coffeescripts
      require 'sprockets'
      output_path     = Rails.root.join("spec/javascripts/assets/")
      env             = Sprockets::Environment.new(Rails.root)
      test_suite_name = "jasmine_spec_suite.js"
  
      test_suite_path = Rails.root.join("spec/javascripts/")
      env.append_path(File.join(test_suite_path))
  
      puts "Compiling JS Test Suite '#{test_suite_name}'."
      compiler = Sprockets::StaticCompiler.new(env,
                                               File.join(output_path),
                                               [test_suite_name],
                                               :digest => nil,
                                               :manifest => false)
      compiler.compile
    end
  

Setup Teamcity for Jasmine Testing

• Create a new build step to compile all coffeescript src files to one big application.js at the build server.
  

  

  
  New build step: Runner type: Rake
  Rake task: projectname:jasmine_asset_precompile
  Additional rake command line parameters: RAILS_ENV=test RAILS_GROUPS=assets
• New build step: JS Jasmine tests
  Command executable: phantomjs
  Command parameters: script/teamcity/test_runner.js teamcity.build.workingDir

By passing the teamcity.build.workingDir to the test_runner.js in the build step configuration (s.a.) we can use the jasmine gem installed in our Rails app from within our TeamCity installation without the need to install a separate jasmine instance on our TeamCity server.

Now for a real example. Let's quickly add a dummy test /spec/javascripts/jasmine_example_spec.js.coffee to our project to break our build:

describe 'A jasmine example suite', ->
  it 'should obviously fail', ->
    expect(true).toBe false # should fail

Do not forget to add the new spec file to your jasmine meta suite /spec/javascripts/assets/jasmine_spec_suite.js and commit the changes.

TeamCity detects the changes and the next available build agent starts testing. Few moments later TeamCity turns red. That's what we expected!

We don't like red builds. Let's fix it!

describe 'A jasmine example suite', ->
  it 'should work flawlessly', ->
    expect(true).toBe true

Commit the file, wait some moments and watch how TeamCity magically changes from red to green. Awesome! Due to the good integration of the JS test reporter you can use the statistics, test history and similiar features of TeamCity as you are used to for your JavaScript tests.

We now have a working continuous integration system for jasmine tests in Rails applications on a TeamCity Server. We get detailed reports for test results - not only for classical RSpec tests but also for Jasmine specs written in plain Javascript or Coffeescript.

One problem still waits to be solved: If jasmine specs fail they're correctly reported as "Failed Tests". However TeamCity doesn't show a stacktrace. If you have an idea how to improve the output please leave a post.

Setup buildagents for CasperJS Testing

Setup Teamcity for CasperJS Testing

• Create a new build step in TeamCity to execute the Casper JS tests:
  New build step: Runner type: Rake
  Step name: Run CasperJS Tests
  Rake task: projectname:run_casper_js_tests
  Bundler: [x] bundle exec

When TeamCity executes this Rake task it creates an XML result file in the project log dir named 'casper_spec_output.xml'. All we need to do to finish our configuration is to add a build step to our TeamCity project configuration to evaluate this file:

Build Configuration --> Build Steps --> Additional Build Features --> XML report processing --> Add build feature
Report type: Ant JUnit
Monitoring rules: log/casper_spec_output.xml
Verbose output: [x]
Further information: http://confluence.jetbrains.com/display/TCD7/XML+Report+Processing

That's it. Let's add an example (failing) CasperJS test (written in CoffeeScript) to our project and commit it: We create a file /spec/javascripts/casperjs/dummy_spec.js.coffee which opens Google in a Browser (PhantomJS).

casper.start "http://www.google.de/", () ->
  @test.assertTitle  "Bing",                   "Google is called Bing!?" # This should fail
  @test.assertExists "form[action='/search']", "Found Google main form"

casper.run () ->
  @test.done 2

Commit it and watch your continuous integration system work for you. Our buildagents execute the CasperJS tests on commit - TeamCity reports the results and turns red!

Next let's fix the test. Change the code and commit it:

casper.start "http://www.google.de/", () ->
  @test.assertTitle  "Google",                 "Google is called Google"
  @test.assertExists "form[action='/search']", "Found Google main form"

casper.run () ->
  @test.done 2

As always TeamCity detects the change and the next available build agent starts testing. TeamCity turns from red to green! Well done!

Conclusion

Remember: You are not limited to CasperJS and Jasmine. There are so many more!

Der Autor Michael Johann (nun wahrlich kein Unbekannter in der Java-Welt) hat seine eigenen Umstiegserfahrungen sehr umsichtig und zielgruppengerecht aufbereitet. Dabei setzt er bei den Lesern einige Grundkenntnisse zu Java und Entwurfsmustern voraus. Das ist aber absolut in Ordnung, denn ganz ohne Erfahrung entscheidet sich wohl kein Programmierer für Rails.

Als Java-Entwickler ist man von den IDEs verwöhnt und möchte bei einem Umstieg möglichst keinerlei Komfort verlieren. Michael Johann schafft einen fundierten Rundumblick durch die Ruby-IDE-Landschaft, gibt Hinweise zur Installation auf verschiedenen Betriebssystemen und vermeidet dabei jedes Anzeichen von Glaubenskrieg. Wer unter Windows entwickeln mag, fühlt sich ebenso gut aufgehoben, wie Linux-Anhänger oder Mac-OS-Jünger.

Agile Entwicklung mit Rails hat ihren Preis: von uns Rails-Programmierern wird allerhand verlangt, was ja durchaus etwas Gutes ist. Und so hat natürlich auch dieses Buch eigene Kapitel zur Testgetriebenen Entwicklung und zur Versionsverwaltung (im Buch leider als Versionskontrolle bezeichnet). Überraschend war für mich (der an der Übersetzung der Pragmatischen Bücher zu CVS, JUnit und Projektautomatisierung beteiligt war), dass das Thema Automatisierung kein eigenes Kapitel erhalten hat. Dabei ist doch gerade in Rails so viel automatisiert, dass mancher Einsteiger eine Referenz "Was generiere ich wie" wünschen würde. Natürlich steht trotzdem genug zur Automatisierung im Buch, man muss es sich eben z.B. aus dem Einführungsbeispiel herauslesen.

Besonders angenehm fällt auf, dass auch die Integration von Java-Teilen eine Rolle spielt. Insbesondere für langjährige Java-Entwickler ist es wichtig, auch zukünftig auf bekannte (und bekanntermaßen solide) Lösungen zurückgreifen zu können. So bekommt JRuby immerhin ein eigenes Kapitel, aber es wird auch an anderen Stellen immer wieder darauf hingewiesen.

Fazit: Dieses Buch ist für Techniker. Es läßt sich gut lesen, denn der Ton ist angenehm sachlich und ohne Schnörkel. Das macht es zu einer klaren Kaufempfehlung für alle, die den Schritt von Java zu Rails gehen wollen. Aber auch wenn man exclusiv bei Java bleiben will, kann sich hier gut darüber informieren, was einem entgeht.

1. JRuby herunterladen von http://jruby.codehaus.org
   (aktuell JRuby 1.1.6: download)
2. Die Zip-Datei in den gewünschten Ordner entpacken.
3. Den Pfad zu JRuby als Systemvariable bekanntmachen.
   (in den Systemeigenschaften -> Erweitert z.B. JRUBY_HOME="C:\Programme\jruby-1.1.6" eintragen.)
4. Die %JRUBY_HOME%\bin an die Pfad-Variablen anfügen.
   (Zum Test, kann in einer neuen Eingabeaufforderung "PATH" eingeben werden.)
5. Nun sollte Folgendes ausgeführt werden können:
   

   jruby -v 

jirb
irb(main):001:0> java.lang.System::out.println("Ein erster Test")
Ein erster Test
=> nil

jruby -S gem list --local

1. Da es sich bei dem Rails Framework um ein RubyGem handelt kann man es sehr einfach installieren.
   

   jruby -S gem install rails --include-dependencies

2. Nachdem Rails installiert ist, kann mittels

   jruby -S rails ersterTest

   eine erste Test-Anwendung erzeugt werden.
3. In das neue Verzeichnis wechseln.
   

   cd ersterTest

4. Der integrierte Ruby WEBrick Server [http://www.webrick.org/] kann mit
   

   jruby script/server
   => Booting WEBrick...
   => Rails 2.2.2 application started on http://0.0.0.0:3000
   => Ctrl-C to shutdown server; call with --help for options
   [2009-02-16 17:30:55] INFO  WEBrick 1.3.1
   [2009-02-16 17:30:55] INFO  ruby 1.8.6 (2008-12-17) [java]
   [2009-02-16 17:30:55] INFO  WEBrick::HTTPServer#start: pid=5716 port=3000

   
   gestartet werden
5. Im Browser http://localhost:3000/ öffnen.
   Vola!