5: End to end testing

Part 5 of our tutorial, in this section we implement end-to-end testing using protractor and astrolabe to create modularised tests.

If you have dropped into the middle of the tutorial you can find the code from the previous step in this tutorial at github:PaulL:tutorial_4, or you can find those tutorial pages either from the index page, or by hitting the tutorials menu in the menu bar above.

At the end of this tutorial we expect to have end-to-end unit testing working, which should give results like this:

e2e test runner

As noted in the previous page, we want to implement end-to-end testing.  Our medium term vision is that we’ll have “vertical” tests, which focus on verifying each of our modules (clubs, teams) work in isolation, connect to their rails backend, and test the functionality of these modules quite thoroughly.  We’ll also have “horizontal” tests that string together across our entities, and verify that our flows work from end-to-end.

We see on the angularJS end to end testing page that protractor is about to become the standard. So, we’ll follow the instructions from the protractor page to use protractor.

To run protractor we need selenium, and there are three parts to that:

  1. Protractor itself
  2. The selenium jar file, which provides a server that can drive your browsers
  3. The chromedriver extension, which allows you to drive chrome directly, without the selenium jar file

Conveniently protractor provides scripts for all this, so install protractor using npm:

sudo npm install -g protractor

Then run the install script for selenium and chromedriver:


Your script location is likely to vary depending on what sort of machine you have – but the npm install should have told you where it installed protractor to.  Try running a selenium server, you should see a clean startup:


If you have protractor 14.0 (released early December), you can instead run:


This will install globally, you can run your selenium server with:

webdriver-manager start

Kill that, and move on to our protractor configuration file.  Documentation and an example can be found on github.  Create a directory ./protractor, and in that directory a configuration file ./protractor/protractor-scenario.tpl.js

exports.config = {
  // ----- How to setup Selenium -----
  // There are three ways to specify how to use Selenium. Specify one of the
  // following:
  // 1. seleniumServerJar - to start Selenium Standalone locally.
  // 2. seleniumAddress - to connect to a Selenium server which is already
  //    running.
  // 3. sauceUser/sauceKey - to use remote Selenium servers via SauceLabs.

  // The location of the selenium standalone server .jar file.
  seleniumServerJar: './selenium/selenium-server-standalone-2.37.0.jar',
  // The port to start the selenium server on, or null if the server should
  // find its own unused port.
  seleniumPort: null,
  // Chromedriver location is used to help the selenium standalone server
  // find chromedriver. This will be passed to the selenium jar as
  // the system property webdriver.chrome.driver. If null, selenium will
  // attempt to find chromedriver using PATH.
  chromeDriver: './selenium/chromedriver',
  // Additional command line options to pass to selenium. For example,
  // if you need to change the browser timeout, use
  // seleniumArgs: ['-browserTimeout=60'],
  seleniumArgs: [],

  // If sauceUser and sauceKey are specified, seleniumServerJar will be ignored.
  // The tests will be run remotely using SauceLabs.
  sauceUser: null,
  sauceKey: null,

  // The address of a running selenium server. If specified, Protractor will
  // connect to an already running instance of selenium. This usually looks like
  // seleniumAddress: 'http://localhost:4444/wd/hub'
  seleniumAddress: null,

  // The timeout for each script run on the browser. This should be longer
  // than the maximum time your application needs to stabilize between tasks.
  allScriptsTimeout: 11000,

  // ----- What tests to run -----
  // Spec patterns are relative to the location of this config.
  specs: [

  // ----- Capabilities to be passed to the webdriver instance ----
  // For a full list of available capabilities, see
  // https://code.google.com/p/selenium/wiki/DesiredCapabilities
  // and
  // https://code.google.com/p/selenium/source/browse/javascript/webdriver/capabilities.js
  capabilities: {
    'browserName': 'chrome'

  // ----- More information for your tests ----
  // A base URL for your application under test. Calls to protractor.get()
  // with relative paths will be prepended with this.
  baseUrl: 'http://localhost:3000',

  // Selector for the element housing the angular app - this defaults to
  // body, but is necessary if ng-app is on a descendant of   
  rootElement: 'body',

  // A callback function called once protractor is ready and available, and
  // before the specs are executed
  // You can specify a file containing code to run by setting onPrepare to
  // the filename string.
  onPrepare: function() {
    // At this point, global 'protractor' object will be set up, and jasmine
    // will be available. For example, you can add a Jasmine reporter with:
    //     jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter(
    //         'outputdir/', true, true));

  // The params object will be passed directly to the protractor instance,
  // and can be accessed from your test. It is an arbitrary object and can
  // contain anything you my need in your test.
  // This can be changed via the command line as:
  //   --params.login.user 'Joe'
  params: {
    login: {
      user: 'Jane',
      password: '1234'

  // ----- Options to be passed to minijasminenode -----
  // See the full list at https://github.com/juliemr/minijasminenode
  jasmineNodeOpts: {
    // onComplete will be called just before the driver quits.
    onComplete: null,
    // If true, display spec names.
    isVerbose: false,
    // If true, print colors to the terminal.
    showColors: true,
    // If true, include stack traces in failures.
    includeStackTrace: true,
    // Default time to wait in ms before a test fails.
    defaultTimeoutInterval: 30000

If you have protractor 14.0, then you need to point to a global selenium and chromedriver, so those lines would be:

  seleniumServerJar: '/usr/local/lib/node_modules/protractor/selenium/selenium-server-standalone-2.37.0.jar',
  chromeDriver: '/usr/local/lib/node_modules/protractor/selenium/chromedriver',

This is telling protractor that we want it to start it’s own selenium server, and we want to run all the files under the src directory that end in scenario.js.

We also need to make some adjustments to our karma setup to have our scenarios not break our karma tests.  Change karma/karma-unit.tpl.js to add an additional exclude line:

    exclude: [
+     'src/**/*.scenario.js'

We also don’t want the scenario files to make their way into our production code – when we run grunt build we want it to exclude anything with scenario in it.  So we modify our build.config.js file to exclude scenario files:

  app_files: {
    js: [ 'src/**/*.js', '!src/**/*.spec.js', '!src/**/*.scenario.js', '!src/assets/**/*.js' ],

We’re also going to modularise our tests somewhat, so we’re going to install the astrolabe extension to protractor:

npm install --save-dev astrolabe

There are a few ways to tie to elements on a page when doing UI level testing.  Protractor provides lots of ways to find elements that have angular bindings on them, but for elements that don’t have bindings the easiest in my experience is to just put ids on the elements.  This means changing your code to make it more testable, but I think that’s a small tradeoff, and there are reasons to put ids on your elements anyway irrespective of testing.

So, update the clubs.tpl.html to have ids on elements we might care about:

  <h1 id="title">Club functions</h1>
  <p id="description">Soon this will show a list of all the clubs, based on information from the server</p>

      <td>Contact Officer</td>
    <tr ng-repeat="club in clubs">

Similarly update index.html, putting an id on the club list item:

                <li id="clubsLink" ui-route="/clubs" ng-class="{active:$uiRoute}">

We’ll start off by using astrolabe to write some partials to represent our page objects.  Firstly, we’ll write the home page, so create a file src/app/home/home.part.scenario.js:

 * Partial for the page objects associated with home
var Page = require('astrolabe').Page;

module.exports = Page.create({
  url: { value: 'UI/index.html' },
  clubsLink: { get: function() { return this.findElement(this.by.id('clubsLink')); } }  

The documentation for astrolabe helps somewhat with the format here, we’re declaring our page to have a URL, and a clubLink field that we care about. We have no helper methods that we need on this page.

Next, we’ll similarly declare a partial for the clubs page, so create a file src/app/club/clubs.part.scenario.js:

 * Partial for the page objects associated with clubs
var Page = require('astrolabe').Page;

module.exports = Page.create({
  url: { value: 'UI/index.html#/clubs' },
  title: { get: function() { return this.findElement(this.by.id('title')); } },
  description: { get: function() { return this.findElement(this.by.id('description')); } },
  clubTableElement: { value: function(rowNum, columnBinding) { 
    return this.findElement(this.by.repeater('club in clubs').row(rowNum).column(columnBinding)); } }

This page has a little more of interest about it, it has again a url that we can use to get to it directly, and it has a couple of fields we care about. It also has a convenience method that allows us to access the cells of our table and look at them.

Lastly, we’ll write a test for the clubs page, create a file src/app/club/club.scenario.js:

 * End to end tests for the club functionality

var homePage = require('../home/home.part.scenario.js');
var clubsPage = require('./clubs.part.scenario.js');

describe( 'Navigate to club list page', function() {
  it ( 'should allow navigation to the club list page', function() {



    expect(clubsPage.title.getText()).toEqual('Club functions');
    expect(clubsPage.description.getText()).toEqual('Soon this will show a list of all the clubs, based on information from the server');
    expect(clubsPage.clubTableElement(0, 'name').getText()).toEqual('First club');    
    expect(clubsPage.clubTableElement(0, 'contact_officer').getText()).toEqual('A Person');    
    expect(clubsPage.clubTableElement(1, 'name').getText()).toEqual('Second club');    
    expect(clubsPage.clubTableElement(1, 'contact_officer').getText()).toEqual('J Jones');    

  it ( 'should allow us to go directly to the club list page', function() {

    expect(clubsPage.title.getText()).toEqual('Club functions');
    expect(clubsPage.description.getText()).toEqual('Soon this will show a list of all the clubs, based on information from the server');
    expect(clubsPage.clubTableElement(0, 'name').getText()).toEqual('First club');    
    expect(clubsPage.clubTableElement(0, 'contact_officer').getText()).toEqual('A Person');    
    expect(clubsPage.clubTableElement(1, 'name').getText()).toEqual('Second club');    
    expect(clubsPage.clubTableElement(1, 'contact_officer').getText()).toEqual('J Jones');    

The matchers and functions you can use with protractor are described in the protractor api docs, although it would be fair to say that the protractor documentation isn’t as clear as you might like yet.  We’ll build these tests out a bit as we go, but ultimately you’ll probably need to fight with the doco.

The code for this section of the tutorial can be found on gitHub:PaulL:tutorial_5.  In the next tutorial we’ll change our list to be an ngGrid, and create an edit page.


9 thoughts on “5: End to end testing

  1. Pingback: AngularJS and Rails 4 CRUD application using ng-boilerplate and twitter bootstrap 3: Tutorial Index | technpol

  2. Pingback: 4: Adding a basic list using $resource for a restful query from rails | technpol

  3. Figured out myself, just ran:
    protractor protractor/protractor-scenario.tpl.js

  4. I’m using a grunt-based workflow. So I run it by:
    grunt e2e

    That functionality comes with ngbp, but I had to extend a little to get it all working.

    For the tests above, you can run them directly with
    protractor protractor/protractor-scenario.tpl.js

  5. Pingback: Automated E2E Testing in Angularjs using Protractor | Scanova Engineering

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s