Protractor and custom failure messages from Jasmine expect()

In this edition I use a custom matcher to provide tailored failure messages on my Jasmine expect() statements, allowing me to get context information other than “expected ‘true’ to equal ‘false'”.

I’ve been working to flesh out the functional tests for our application – I had allowed them to fall a little into disrepair whilst I did some architectural change, and it was time to refresh the suite.

As part of my architectural changes on the application itself I had refactored a lot of code to make it common.  Looking at the functional test suite it looked like it needed the same treatment.  In refactoring this code I wrote some helper routines, including one that checked (for each of view, editClean, editDirty modes) whether the correct fields and buttons were enabled.  All well and good, but the core of that logic was a loop through a hash, and Jasmine/Protractor reports only the line number – not which iteration through the hash we were on.  Making debugging annoying – how do we tell which field it is that is enabled but shouldn’t be?

So, my code looked like this:

pageObject.checkObjectEnablement = function( expectedMode, objectHash ) {
  var key;
  
  for( key in objectHash ) {
    var myElement = element( by.id(this.name + '.' + key ));
    var elementName = key; 

      switch( objectHash[key][expectedMode] ) {
        case 'enabled':
          expect( myElement.isDisplayed() ).toEqual( true );
          expect( myElement.isEnabled() ).toEqual( true );
          expect( myElement.getAttribute('readonly') ).toEqual( null );
          break;
        case 'readonly':
          expect( myElement.isDisplayed() ).toEqual( true );
          expect( myElement.isEnabled() ).toEqual( true );
          expect( myElement.getAttribute('readonly') ).toEqual( 'true' );
          break;
        case 'not_present':
          expect( myElement.isPresent() ).toEqual( false );
          break;
        default: 
          error( 'at least one element in the element array has a mode that isn\'t in the list enabled, readonly, not_present' );
          break;
      }
    }
  }       
};

I would then call this from the test script itself:

it ( 'fields are enabled as expected', function() {
  projectPage.checkObjectEnablement( 'newDirty', projectPage.projectFieldsHash );
});

An expectation fail leads to the error message "expected 'false' to equal 'true'" and a stacktrace giving the line number, which isn’t overly helpful for debugging.

I looked around the interwebz and found this discussion, relating to a toolset that uses Jasmine, and making mention of a .because syntax that would allow you to add a custom explanation to a failure.  But that request is pretty old, and it looked like there wasn’t likely to be progress any time soon.

I looked around further, and found two potential ways to resolve this.  The first one is to just push the “it” clause into the helper function itself. You can dynamically build the it clause, so you could get a better contextualised message that way. You’d end up modifying the function something like the below:

pageObject.checkObjectEnablement = function( expectedMode, objectHash ) {
  var key;  
  for( key in objectHash ) {
    it( 'expect ' + objectHash[key].name + ' to be ' + objectHash[key][expectedMode] + ' when in mode: ' + expectedMode', function () {
      var myElement = element( by.id(this.name + '.' + key ));

        switch( objectHash[key][expectedMode] ) {
        ....
        ....
        }
      }
    });
  }       
};

I think this would work, but I generally don’t see people nesting if blocks in Jasmine – you have nested describes, then the bottom level is an it. I didn’t want a situation where in my test spec I had to know whether or not a helper function I called had an “it” within it so I could work out whether to call it from a describe or from an it, it would make things less intuitive.

The alternate that I found was to write a custom matcher, which lets you have a custom error message on failure. The disadvantage of doing that is that I’d imagine that I’ll miss some magic that might be in the standard Jasmine matchers – maybe miss some corner cases or something. But when I’m only checking booleans I can probably deal with that – it seems like I can work out how to compare two booleans adequately.

This is working well for me. First you define the custom matcher somewhere nice and central:

beforeEach(function() {
  var matchers = {
    toEqualBecause: function( value, message ) {
      this.message = function() {
        return "Expected '" + this.actual + "' to equal '" + value + "' because " + message;  
      };
      
      return this.actual === value;  
    }
  };

  this.addMatchers(matchers);
});

Then you can modify your expectations to use the new matcher:

pageObject.checkObjectEnablement = function( expectedMode, objectHash ) {
  var key;  
  for( key in objectHash ) {
    var myElement = element( by.id(this.name + '.' + key ));
    var elementName = key; 

    switch( objectHash[key][expectedMode] ) {
      case 'enabled':
        expect( myElement.isDisplayed() ).toEqualBecause( true, 'element ' + elementName + ' has a config of enabled, and should be displayed' );
        expect( myElement.isEnabled() ).toEqualBecause( true, 'element ' + elementName + ' has a config of enabled, and should be enabled' );
        expect( myElement.getAttribute('readonly') ).toEqualBecause( null, 'element ' + elementName + ' has a config of enabled, and should not be readonly' );
        break;
      case 'readonly':
        expect( myElement.isDisplayed() ).toEqualBecause( true, 'element ' + elementName + ' has a config of readonly, and should be displayed' );
        expect( myElement.isEnabled() ).toEqualBecause( true, 'element ' + elementName + ' has a config of readonly, and should be enabled' );
        expect( myElement.getAttribute('readonly') ).toEqualBecause( 'true', 'element ' + elementName + ' has a config of readonly, and should be readonly' );
        break;
      case 'not_present':
        expect( myElement.isPresent() ).toEqualBecause( false, 'element ' + elementName + ' has a config of not_present, and should not be present' );
        break;
      default: 
        error( 'at least one element in the array has a mode that isn\'t in the list enabled, readonly, not_present' );
        break;
      }
    }
  }       
};

Upon a failure you’ll then get a message like:
Expected 'false' to equal 'true' because element create_user has a config of readonly, and should be displayed

Advertisements

7 thoughts on “Protractor and custom failure messages from Jasmine expect()

  1. From paul scott NZ
    not flippant PaulL Can you get a program to allow people to take emails from Web, to download on hard drive . thereby remove hack threat. You could call it a “Predator drop down ”
    Many people need protection from internet invasion.
    Your photos on headings are good, very good for an Australian, but some could do with higher f stops yet.

  2. Paul:

    The photos came with the theme – I can’t take credit for them. I quite like them though.

    Google appear to have an export feature: http://gmailblog.blogspot.co.nz/2013/12/download-copy-of-your-gmail-and-google.html. In theory that means you just export your Gmail, then delete it all from GMail so that people can’t see it.

    An alternative approach is to use a mail package that allows you to create a “local mailbox” on your computer. You can then just drag all the mail you no longer want in Google into that local mailbox, it should disappear from GMail (check by logging into the web version to see), but would remain available to you in your local mailbox, so you can still find it and look at it if you need it.

    In general I think the main risk with a GMail hack is that many (most?) people federate a bunch of stuff to their GMail logon – so if someone cracks your GMail they can:
    – log on to a bunch of websites that were secured with your google password
    – issue password resets and pick up the link from your GMail, thereby gaining access to sites you are a member of
    – sell your identity

    This is usually more important stuff than them reading your e-mail, I guess depending on how embarrassing the contents of your e-mails are.

    If you’re worried about that, a better approach might be to harden your security configuration:
    – turn on Google 2 factor authentication – it means that your GMail can only be accessed from authorised devices, and the process for authorising a device requires more than just the password
    – make sure the password for different websites is different – don’t share passwords
    – use a password locker like lastpass.com, and use a strong password on lastpass. Then you don’t have to remember all those passwords, but you have strong security around them.

    The reality is that all this security comes at some cost in usability. I generally do a mental risk assessment of whether someone would deliberately target me, or whether they’d just be doing a general sweep. If just a general sweep, then basic security precautions are enough. If someone’s directly targeting you then you need to up it a bit – but I don’t think anybody would want to directly target me.

    I guess since I wrote all this I may as well cross post at Kiwiblog. 🙂

  3. excellent PaulL, yes please do post on Kiwi , unlike me you have a pretty regular name there,
    I mean you know dependable wisdom politically.
    Also you are an expert on those J curves
    I tell my Thai maid , you gotta stop sending me sex emails , its one thing to lie down and sleep between me and wife, with your girl J curves on, but another another to tantalise on e-mail with pictures in knickers . God help me if the hackers find out.

  4. Just thump them PaulL. what is wrong with our people that they can not have secure email accounts.

  5. I did it in a similar form for my own too and for jasmine 2.0 and the related definitely-typed and I’ve posted it here: https://github.com/davidemannone/jasmine2.0-explained
    Here is how to use it: expect(SOMETHING).toEqual(WHAT-EXPECTED).byFailReport(“YOUR-CUSTOM-REPORT!”);
    and this reports only in case of the match fails this: “Expected SOMETHING to be WHAT-EXPECTED. >YOUR-CUSTOM-REPORT!<
    For more details read the Readme.Me file.

  6. Your first method seems like it would work really well for running quick verification on fragments used within page objects, and abstracting away checking every single element in the spec file.

  7. yeah, that was my intention. I wanted to have a config file/matrix for each page that listed the elements and whether they were mandatory, enabled or not in given modes (so maybe enabled in write mode, disabled in read mode, some other state if using role based security etc). Then have a single “check page state” function that just drove off the config file. Then writing my test scripts was mostly about filling in the config matrix, rather than writing screeds of code.

    Of course, this was a while ago now, and I’m not working on that project any more. 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s