Bridging the Gap – Automation

A few months ago I was mulling over the best way to approach a proposed automation initiative I had proposed when I came across a well written and highly informative blog by Elisabeth Hendrickson that helped me focus on my objectives and narrow my search criteria. Our situation is largely similar to a lot of other organisations in that we have a web-based product with a plethora of checks that are ripe for automation. We’re understaffed and overworked (aren’t we all?), so my primary aims included:

• A way of executing automated functional checks on existing features quickly for each candidate build
• Transparency across platforms and browsers (Windows, MacOS, Firefox, Chrome, IE, etc.)
• Allow the sapient testers to focus on testing and discovering new information about the product
• An efficient way of communicating automation results to all stakeholders without overwhelming them with data
• A way of involving the non-technical stakeholders so they feel part of the technical side of things (without having to be technical)
• Providing a way of capturing screenshots that allow manual inspection of aesthetics (also useful for demos, presentations, bug reports, etc)
• Build some bridges between technology-business, hopefully stimulating further collaboration through visibility

Implementation

I’m a big fan of scripted languages and tend to go for Perl more often than not, but Ruby was catching my eye for this particular adventure. I love the ability to simply  “write’n’run” code without having to deal with compilation, linking, etc or having to invest in expensive IDEs to bring projects to life. Ruby has elegant syntax and a wealth of support in the online community, plus the inclusion of IRB (Interactive Ruby Shell) which was marvellously easy to use when experimenting with new code.

I just wanted an intelligent editor and a command line. As far as editors go, vim  was always an option but as I was going to implement this on a Mac, I plumped for BBEdit which, as Bare Bones Software suggests, “it doesn’t suck”. This afforded me the ability to switch between multiple files very quickly (you can do this with vim too, but I liked the presentation and the reviews were favourable).

Drivers

While the implementation was a no-brainer, the choice of driver – i.e. the thing that would manipulate my web interface was swayed somewhat by my previous experiences with Selenium. Like most people, I had used the Firefox plugin and played with the IDE record/playback functionality and found that the export utility to Ruby was less than perfect. Selenium RC (as it was called at the time) wasn’t much better (for me at least) and I encountered some real problems when trying to interact with javascript elements. It’s much improved now by all accounts but I was already searching for alternatives (a bias on my behalf, I guess).

The other obvious choice was Watir which seemed like a natural fit to me, but after reading a series of posts from Alister Scott, I chose to go with Watir-Webdriver instead. I particularly enjoyed his ‘meaty example‘ post, comparing Ruby automated APIs. The syntax looked elegant as well as affording me the chance to make the code maintainable (as well as readable). In principle, this choice would support IE, Firefox and Chrome across Windows and MacOS (with the possibility of Opera too). It also allowed me to capture ‘full screen’ screenshots (as opposed to ‘visible area’ screenshots) – something which my previous endeavours with Selenium caused more pain by not working as I’d hoped.

Frameworks

There are a number of frameworks to choose from but the first couple I looked at were a bit too ‘codey’ for my non-technical stakeholders, namely Test::Unit and RSpec. I even bought a copy of The Rspec Book but in doing so ended up reading and digesting the chapters within that talked about Cucumber. A soon as I saw this I knew it would give me the best chance of drawing in the non-technical stakeholders. I ended up buying The Cucumber Book as well, but not until I was most of the way through implementation – thankfully there are lots of great online references, tutorials, and examples to learn from on the web. The key things about Cucumber that won me over were:

• Plain text descriptions
• Nifty results presentation with the default Cucumber HTML formatter
• Even niftier results presentation with a customised Cucumber HTML formatter
• Ability slice and dice the automation suite using Cucumber tags – a very powerful feature

Now, granted we don’t do any sort of BDD at our place, but that didn’t stop me from choosing Cucumber (nor should it, IMHO)

Move to the current day, the initiative is complete and has been well received so far.

I’ll elaborate more on the details in later blog posts.