The Testing-Checking Synergy

Like many organisations out there, we have witnessed a surge in automation over the last few years with a multitude of tools, frameworks, languages, etc. as we drive towards zero-click deployments for the plethora of squads/tribes within the business. During this time, I’ve slowly but surely witnessed talk about testing and automation/checking become synonymous (primarily from non-testers) as test-personnel within the organisation spend more and more time focused on writing/refining automated checks within their suites.

Some years ago when agile started becoming mainstream, there were proponents who were extremely vocal and forceful about agile practice (I recall Michael Bolton referring to them as Agilistas), almost to the point where they seemed somewhat rabid. I was all for someone getting excited about something positive, but by the same token I firmly believed they had to CALM DOWN!

The reason I mention this is because I see some resemblance in the type of human behaviour surrounding automation nowadays. Particularly from non-testers, where similar frenzied (almost fanatical) endorsements of existing automation (irrespective of how good/bad/fast/slow/thoughtful/shallow) and calls for more, MoRe, MORE! automation is underpinned by wild gesticulation, loud voices and (in some cases) frothing mouths.

The concern I have of course is that amidst the frenzy about automation is that the distinction between testing and checking is misunderstood/forgotten/ignored, supplanted by the belief that everything valuable about testing is embedded within the automation artefacts  that they lavish praise on.

I recently got a chance to present into our VP a while back and I wanted to have something up my sleeve that clearly illustrated that testing and checking were not the same thing, but also to place the importance one has on the other. I also wanted to show what would happen if you took testing out of the equation.

Dan Ashby’s recent blog rang a few bells for me when he cleverly related activities to information. This was enhanced further by John Stevenson, who also offered another unique model after MEWT 4 last year. After borrowing from both I adapted it to my own needs – something which I think nicely illustrates the synergy between testing and checking, while at the same time, showing that they fall on different parts of the “information spectrum”:

testing-checking

As you can see I’ve borrowed from the former US Secretary of Defence, Donald Rumsfeld too, and while I’m no real fan of his I do like the fact his somewhat famous epistemology (“there are known knowns, known unknowns and unknown unknowns”) applies so well to testing. One could say that the art of checking lies firmly in the known knowns domain, while testing lies in the known unknowns and unknown unknowns domains.  More importantly, if you take testing out of this equation, here’s what I believe you’re left with:

notesting-checking

If you’re not testing, you’re not uncovering any information, and in your not uncovering any information, you’re simply confirming nothing more than speculation about what your product may (or may not) do, and there’s an enormous amount of information that you don’t (and may never) know about your product.

Can we really afford to focus on checking without doing any testing whatsoever?

I don’t think so.

RAZOR – A Webpage Testing Heuristic

Last week, Richard Bradshaw posed a mini testing question on a slack channel, asking whether anyone could reproduce a particular problem with an O2 webpage (see below) which has since been fixed. I’ve seen this type of issue quite frequently and had a hunch about how it could be reproduced, and as it turned out my hunch was good. However, it was only when Richard asked me about why I tried a particular course of action that I realised that I had a bespoke heuristic tucked away in the back of my mind that I hadn’t given a name to, so here it is. My quick and easy heuristic for testing webpages:

RAZOR

  • RResize
  • AAccessible
  • ZZoom
  • OOptimisation
  • RRefresh

Resize – Is the page responsive? How does it handle being resized to minimum viewable dimensions (and beyond)? What about maximum dimensions (for your display resolution)? What if you go beyond that and spread it across multiple monitors?

Accessible – How accessible is the webpage? Need some help? Give it a little wave using the WAVE (Web Accessibility) Evaluation Tool for Chrome

Zoom – A much overlooked manipulation and in the example above, how I found the issue originally highlighted by Richard. I tend to use this habitually (even when not explicitly testing something), especially when I’m using my 13” MacBook display and screen real estate is at a premium.

Optimisation – Is the page displayed optimally for the device you’re viewing it on? No need to break your multitude of devices just yet – try using something like the User Agent Switcher to get a quick feel for how the page renders on other devices.

Refresh –  Is the webpage layout identical when you refresh it? What about a force refresh (where you force a refresh of your browser’s cache)?

As with all heuristics, the above is of course fallible, but hopefully you’ll find it easy to remember as well as useful.

O2-webpage

PS: The above was achieved by zooming to 90% (and has been fixed)

RapidReporterMac Now Available

I’ve always been a big fan of Shmuel Gershon’s Rapid Reporter note-taking tool. It’s subtlety and simplicity are the key combination of factors that make it such a success. It doesn’t hog the limelight, it doesn’t over-complicate things, and most importantly, it doesn’t take your attention away from your testing.

As an avid Mac user for many years, I have always lamented the absence of Rapid Reporter for the Mac, so when we had an intake of interns at Skyscanner this summer, I put a very talented chap, Ross Conyers to good use to create an OS X port of this wonderful little tool with nothing more than a MacBook Pro, a copy of XCode, a crash course in Apple’s Swift language (and me providing input for the duration, all with Shmuel’s blessing, of course).

As this endeavour was done on my company’s dime, we opted for distributing this to the masses via Skyscanner’s organisational github account. Source code, binaries, the lot. The github public repository can be found here

As most of you will just be looking to install and use it drop down into the RapidReporterMac folder and download/run the DMG and you should be good to go. We’ve tried to make it look and feel as close as we could to Shmuel’s original and have added some Mac specific keyboard shortcuts (see the README.md in the repo for details).

Happy note-taking, testers!

As easy as riding a bike…

As I mentioned in a previous post, Mark Tomlinson’s presentation about insight from TestBash 2013 mad a big impression on me. Not just because of the wonderfully confounding spinning cat, but because of the neurological process of insight which has the potential to give testers like us the potential to see things that others can’t. The principle of un-learning what you know, relaxing your cortex as means of triggering that process is fascinating, and the other day I stumbled across another great example of having to un-learn something in order to overcome a cognitive bias.

This came from a YouTube video on the Smarter Every Day channel with Destin Sandlin and dealing with what was termed a backwards bike.

This was a bike where the head tube (steering column) had been modified so that turning the handlebars LEFT would turn the front wheel RIGHT and vice versa.

bike

I know what you’re thinking – How hard can it be? Just reverse your thinking! or even better, Don’t turn the handlebars at all, just go straight! Piece of cake! It’s not rocket science, for goodness’ sake.

Well, Sandlin is a rocket scientist and it took him 8 months to learn how to ride the backwards bike, and admittedly even then he still didn’t find it easy. His son on the other hand who was maybe 5 or 6 managed to ride it within 2 weeks.

Much like spinning cat, although most of us could get it to change directions at some point we couldn’t necessarily do it on demand, nor explain what we did to trigger the switch. With the backwards bike you can see that even the slightest distraction as Sandlin was riding it caused him to flip back to his old ways of thinking and wipe out.

One thing I noted about this video is that most people who tried to ride the backwards bike already knew how to ride a normal bike, and encountered the same bias as a result (at least I’m assuming the people on stage in the video already knew how to ride a bike, otherwise why would they offer to get up and ride a backwards bike?) It would have been interesting to see how someone who doesn’t know how to ride a bike would have coped with the backwards bike.

As far as testing goes, this illustrates that simply knowing about your bias doesn’t automatically mean you’ll overcome it, but it is a start.

Testing Mnemonics – Desktop Download

Good news, I have a present for you all. 🙂

Testing mnemonics are not exactly new, but are still fantastic ways of generating ideas for testing.

Earlier this year in her excellent TestBash workshop (“Building an Itinerary for Exploratory Testing”), Karen Johnson introduced us to her “mnemonics card deck“, which was a very unique way of bringing testing ideas to the classroom and/or workplace. You could, of course print Karen’s document, cut out the cards, laminate them (if you were so inclined) and put a keyring through them all, leaving you with a portable and powerful aid right there on your desk.

I took inspiration from this, but instead of reaching for the printer, laminator and hole puncher when I got back to work, I embraced my inner Picasso/Banksy and attempted to create a subtle but visually engaging image that could take pride of place as my desktop background, and provide me with an easily accessible form of testing inspiration. There were ‘economy of space’ considerations of course, but hopefully I’ve struck a balance so it’s subtle, engaging without being overbearing and busy.

The three primary sources I used to compile this were Karen Johnson’s card deck, Lynn McKee’s mnemonics page and the wonderfully inventive posts from the Moolya Blog.

I’ve also provided customised bitly shortened URLs to the mnemonic creator’s original reference, and attempted to be as consistent as I can with the customised/shortened URLs so they’re easy to remember (I generally tried to use the mnemonic appended with ‘_m’ – E.g. http://bit.ly/sackedscows_m). Some older links were outdated and defunct but I’ve updated them to something more contemporary (by the same creator). Any mistakes are entirely mine.

testingMnemonics

Feel free to download the image, distribute and become inspired – available in the following formats:

DOWNLOAD JPG

DOWNLOAD PNG

DOWNLOAD PDF

You can also use the link in the blog menu above (defaults to the JPG).

Enjoy!

TestBash 2015 – Are ye Dancin’?

In years gone by, the culture of Scotland’s dance halls (and in my generation, the discos) involved a time-honoured exchange between a young lad who wanted to dance with a particular young lady. It went something like this:

Him: Are ye dancin’?
Her: Are ye askin’?
Him: I’m askin’!
Her: I’m dancin’!

This was a successful outcome, however, if the young lady in question did not want to dance the initial (blunt) response would be along the lines of Naw, it’s just the way I’m standin’. Ouch!

What does this have to do with TestBash or Testing? Read on….

There’s definitely something to be said for the almost overwhelming feeling of elation coming away from something as inspiring as TestBash. A newly found or renewed sense of irrepressible enthusiasm that provides you with the impetus to own your testing role more than you ever did before. Whatever the catalysts may be for your own personal collection of ‘OMG moments’ – be it the workshops, the speakers, the material, the community, the kick-ass Lego robots, the Desmond Tutu inspired dress-sense, or even simply the alcohol – be sure to embrace this mind-set and carry it with you back into your place of work.

In terms of your place of work and particularly from a software testing perspective, it’s probably fair to say that it can be categorised in one of three distinct ways:

  • It’s progressive and forward thinking
  • It’s stagnant and in dire need of change
  • It’s a bit of both

For those you that work in environments with outmoded attitudes that marginalise software test to some degree, consciously or unconsciously, it’s easy to quickly lose your position on that wave of euphoria you’ve been surfing on for the duration of the post-TestBash weekend, as your work place reality delivers a cold, hard slap in the face with a big wet fish.

If that’s indicative of your situation to some extent, then here’s what you can do to get back on that surfboard and ride that euphoric wave again – watch this video:

Now, you’ve watched it (You have watched it, haven’t you?) – don’t be alarmed – I’m not asking you to start drinking and dancing in the workplace stand-up areas (one could argue, however, doing so could dramatically enrich some agile stand-up meeting experiences)

This video, referred to using a multitude of names such as ‘The Dancing Guy’, ‘The Lone Nutter’, and ‘Crazy Dancing Dude’ has been around for some time and was first used in a TED Talk back in 2010 about how to start a movement, but has since been used in a number of business themed talks regarding leadership, etc.

What I’m suggesting you take from this video (and indeed, blog post) is the commitment to act on the inspiration you took from TestBash, and use it to be an agent of change in your workplace.

Be the dancing guy, the lone nutter, the crazy dancing dude on the hill.

Educate the nay-sayers, give insightful presentations, write blog posts, start discussion forums, utilise social media on your company intranet (if you have such a thing), ambush stand-up meetings, illustrate how testing is not the same as checking, warn about the folly of counting test cases, start internal testing workshops, set tricky testing problems, encourage collaborative debrief to exchange ideas so you can learn from each other, invite developers, engage with them, ask them to explain their unit testing strategy, talk about TDD, consider continuous integration, bridge the gap to your non-technical stakeholders by suggesting BDD as a means of bringing them into the fold, explore options, ask questions, ask LOTS of questions, solicit the opinion of the people who matter, make a difference, have an impact.

Be the dancing guy, the lone nutter, the crazy dancing dude on the hill. You may be on your own to begin with, people may be reluctant to join you at first and simply observe from afar, some may even laugh at you. But slowly and surely, people will begin to come around. Change is difficult and slow for most people, but you’ll eventually reach a point where you’ll achieve critical mass, and suddenly you won’t be the odd ones out any more. All this, fuelled by that post-TestBash euphoria. Get out there and dance!

So, Are ye dancin’?

Dressing Up the Spinning Cat

It’s probably safe to say that most of you will have been aware of, or perhaps even participated in, some form of discussion/debate/deliberation/divorce/dust-up (delete as applicable) over the now infamous two-tone dress that trended worldwide on social media, and in some cases, made national news headlines.

The source of all discussion was down to the dress’s original colour(s)

Is it white & gold, or blue & black?

that-dress

There have since been innumerable articles written about the ‘science’ behind why people see different colours. Endless paragraphs about how light enters our eyes, how different wavelengths correspond to different colours, and how that stimulates neural activity relating to the visual cortex. It’s all very interesting, of course but then something happened that made me think of something else that had slipped my mind.

Rewind 12 months and I’m sitting at TestBash 3 in Brighton, watching Mark Tomlinson present a fascinating piece about insightful testing.  He quotes Donald Rumsfeld as an unlikely source of testing wisdom while teasing us with a sequence of optical illusions, culminating with the superbly divisive pièce de résistance – ‘the spinning cat’ (Clockwise or anti-clockwise?).

The real meat and potatoes of this talk for me was the information about the neuroscience behind it all as described by the paper “The Prepared Mind”. Essentially it’s to do with what we tend to call the ‘Eureka’ moment – left brain/right brain communications, relaxing your cortex, serendipitous connections, new neural networks coming to consciousness (I.e. solving problems, seeing new perspectives, gaining new insights) before you’re even aware of it. Captivating stuff.

How does this relate to ‘the dress’?

Well, I was very firmly of the opinion that it was in fact white & gold, but my wife could only see blue & black. It was only when I was quickly scrolling through some article to quote some scientific reference at the end, the image fleetingly passed my field of vision, but for the very first time the dress appeared to me as blue & black, not white & gold. When I scrolled back it had indeed reverted to white & gold, but on occasions since then when I’ve been relaxing my cortex and not staring directly at the dress, I’ve been able to get it to switch colours again, albeit very briefly.

So if Mark ever plans to give that talk again, perhaps he should consider having the spinning cat wearing that dress to really mess with your heads 🙂

Repeat after me, ‘relax the cortex’……..

Cucumber 101

My primary reason for using Cucumber in a recent automation initiative was to try and bridge a technology-business divide that existed in my workplace by giving the non-technical stakeholders a larger sense of involvement in testing with a (longer term) view to them actively contributing to future test specs by using the plain English approach that Cucumber does so well.

However, when I first started looking into Cucumber, there were a multitude of references out there but as I was also learning about Ruby, Watir and RSpec at the same time it became a little difficult to see the wood for the trees because they tended to talk about all of these things without being clear about where one thing stopped and another began.

What I’d like to do with this post is present a simple Cucumber 101 tutorial, letting cucumber guide us through the creation of a set of ‘scenarios’  (this is what Cucumber calls ‘tests’ or to be more precise, ‘checks’ – this is automated after all).

1. Cucumber Basics

Cucumber is a command line tool and when you run it, it looks for plain-language text files which typically contain one or more features, and within each feature, one or more scenarios to test. Within each scenario you have a number of steps that Cucumber works through. There is some syntax involved so that cucumber can understand what you’ve written and this is known as Gherkin. I’ll highlight this in the examples below.

cuke-stack

So if you were to see the directory structure required for a simple cucumber project, it would look something like this:

cuke-01

2. Installation Prerequisites

This tutorial assumes you have a copy of ruby, rubygems (or equivalent), watir-webdriver, rspec and of course cucumber installed on your target system.

3. Create a Scenario

Let’s create a new Cucumber project (a directory, called ‘Cuke’) and try running cucumber in this empty directory.

Wait, how do we run cucumber? Simple, type ‘cucumber’ (Note: We can specify arguments but in the absence of any cucumber assumes you want to run everything – more on that later).

cuke-02

So we have no features or scenarios. Cucumber is looking for a features directory. Let’s create one as it suggests. Once we’ve done that, let’s try running cucumber again.

cuke-03

So now we see that although cucumber found a features directory, there were no feature files and consequently no scenarios to run, hence the “0 scenarios” and “0 steps” references.

Let’s create a scenario in a feature file under the features directory, and let’s call the feature file karate.feature. Here’s the content:

cuke-04

For those of you unfamiliar with karate, Hironori Ohtsuka was the founder of the Wado-Ryu style of karate, and our scenario is checking for the presence of the founder’s surname in the Google results on a search for ‘Wado’.

You can see the Gherkin grammar highlighted in blue above – these must be used when writing feature files – the key Gherkin words include – Feature, Background, Scenario, Scenario Outline, Scenarios (or Examples), Given, When, Then, And (or But) and a couple of other symbols.

Now let’s run cucumber again with the new feature file in place:

cuke-05

So, cucumber is now picking up our new feature file. Great!

However, all of our scenario steps are as yet undefined.

4. Add Step Definitions

So, two things about the above output. Cucumber is telling us that our steps and scenario are as yet undefined (the yellow text) and that as yet, it doesn’t know which programming language we intend to use to implement our scenarios (the red text). Let’s take our lead from Cucumber and copy the suggested steps into a new step definition file, say step_karate.rb (The .rb suffix indicates a Ruby file). Like so…

cuke-06

and run cucumber again

cuke-07

You may notice that the red text we saw in the ‘undefined’ error above has now gone. Why? We’ve put a Ruby (.rb) step definition file in place so Cucumber now knows we’re intending to use ruby to implement the scenario. So now we can map all the steps in our scenario to a step definition file, and cucumber helpfully provides the file and line number of each step it finds (this comes in useful later when you have large suites of scenarios) You’ll also notice that Cucumber has deemed the first step as ‘pending‘ (I.e. to be implemented) – this is due to the pending statement in the step definition. This acts as a placeholder so you can write some proper code for each step when you are ready. Cucumber takes a sequential approach to scenario steps and the subsequent steps are deemed ‘skipped

5. Implement Step Definitions

Let’s implement our first step definition now. We’ll take out the pending statement and put in some Ruby code that will start a web-browser.

cuke-08

So for our first step “I navigate to Google” we’ve removed the pending statement and inserted two lines of Ruby/Watir-Webdriver, to start a browser (Firefox) and to navigate to Google.

Note: I’ve included a Ruby ‘require’ statement in the step definition file. There are tidier ways of doing this since the key to maintainability of any cucumber suite lies with the step definition files. We’ll tidy this up a bit later

Let’s run cucumber again

cuke-09

Now we see the first step passes (green), the second step is pending (yellow), and the third step is skipped (blue). Progress! You’ll also notice a browser opened in front of you (at Google) that hasn’t been closed – that’s because we didn’t close it in our code (yet). For now, kill the browser – we’ll tidy that up later. Let’s implement the remaining steps as shown below:

cuke-10

In the second step, we create variables for the Google search field and button, then enter the value ‘Wado’ into the search field and click on the button. In the third step we check that the results page has loaded and then perform another check to see if the text  ‘Ohtsuka’ is present in the results page. If it’s there, the step will pass, and if it’s not the step will fail. Lastly, we close the browser instance.

6. Passed & Failed Scenarios

So, if we run cucumber once again with all our implemented steps:
cuke-11
Whoa! 3 passed steps and 1 passed  scenario. Nice!
However, in order to see what a failed scenario looks like, let’s deliberately alter the string literal in the last step  ‘Ohtsuka’ to ‘Octavias’ as we don’t expect that to appear in the page, and run cucumber again:
cuke-12
So you can see from the above the scenario has now failed after being unable to locate ‘Octavias’ in the results page. If you run this you’ll notice that the browser stayed open which means the @browser.close statement was never executed. Hmm, messy. Let’s tidy this up a bit…

7. Tidy Up – Environment, Hooks, etc

First of all, let’s get rid of that ‘require’ statement in the step definition file and put it somewhere more appropriate. We’ll create a new directory under features, called support, and under there we’ll create an environment file, env.rb and add two lines

cuke-13
The first line extends the path that Cucumber picks up to include a Cuke/lib (we haven’t created that yet)
The second line includes a Ruby file called myClasses.rb (we haven’t created that either)
Let’s create those too.
cuke-14
So our require watir-webdriver statement lives here, we can remove it from step_karate.rb
Next, let’s add another file to the support subdirectory, called hooks.rb and use the Before and After methods to define actions that should be taken before and after each and every scenario. Like, starting up and closing down a browser.
cuke-15
Now we can remove those statements from step_karate.rb as well.
If we re-run the scenario this time, it will still fail but the browser will be closed down regardless of the outcome.

8. HTML Formatters

You can also direct cucumber output to  an HTML file using the default HTML formatters. by running the command:

cucumber -f html >output.html

The contents of output.html then look like this:
cuke-16

So as you can see the failed step is highlighted in red, as is the scenario, as is the cucumber header (if one scenario out of many fail then the header is coloured red accordingly).

Now let’s change the string ‘Octavias’ back to ‘Ohtsuka’ and re-run

cuke-17

The Scenario headers in these HTML files can be collapsed or expanded individually (left click on it) or along with everything else in the page (use ‘Collapse All’/’Expand All’ in the header)

9. Multiple (Similar) Scenarios – Placeholders

So our scenario checks the presence of the karate founder’s surname in the results when we search for the style name. What if we wanted to run a similar scenario for other styles of karate, say:
  • Wado-Ryu (Hironori Ohtsuka)
  • Shotokan (Gichin Funakoshi)
  • Goju-Ryu (Chojun Miyagi)
  • Kyokushinkai (Masutatsu Oyama)
We could of course write separate scenarios and create new step definitions (primarily for steps 2  and 3, step 1 could be reused). The trouble there is that when you have a lot of scenarios which do the same thing with different arguments, it becomes laborious to wade through the results and difficult to see the uniqueness of each scenario.
So let’s consider refactoring our scenario to become a scenario outline – that way we only have to specify our steps once, and use placeholders to identify the unique values we wish to use within each iteration. Here’s what the refactored feature file looks like:
cuke-18
So the placeholders are the elements in the angled brackets in the steps, <karateStyle> and <masterSurname>. The Examples section (Note, you could also use the text ‘Scenarios:’ here instead of ‘Examples:’) is simply a table with the placeholder names (minus the angled brackets) as headers and the values you wish to use below them.
The scenario outline is useless without the examples and the four examples now constitute a single scenario.
Let’s take a look at the refactored step definition file:
cuke-19
The (\w+) elements are shorthand character classes and stand for a word character, specifically [A-Za-z0-9]. The value is read into the variable (karate or surname) which we can then use in place of the string literals (‘Wado’ and ‘Ohtsuka’) we had previously.

10. Tagging

Tagging is easily one of the most powerful aspects of cucumber and for me, the thing that made it so versatile when dealing with a large number of scenarios. A tag in cucumber is simply a text string preceded by the ampersand character, and we generally use them in the feature file by placing them before Gherkin keywords such as Scenario Outline, Scenarios, Examples, etc. You can have more than one tag for the same scenario, just separate them with spaces.
Tags can represent anything that’s meaningful to you – the name of the scenarios (e.g. @scenario1, @scenario2, etc.) , the performance of the scenarios (e.g. @quick, @slow, @glacial), the frequency of the scenarios (e.g. @hourly, @daily, @weekly, etc.) – anything that makes it easier to slice and dice your scenarios into meaningful cross-sections that make execution a breeze.
Let’s refactor our feature file with some tags:
cuke-20
We can run cucumber and pass in some arguments to tell it to run everything associated with a particular tag by using a command like

cucumber -t <tag>

where <tag> is the tag name. So in the case above if we ran:
  • cucumber -t @all  (This would run everything – all four scenarios)
  • cucumber -t @daily (This would run the top two scenarios for Wado and Shotokan)
  • cucumber -t @monthly (This would run the bottom two scenarios for Goju and Kyokushinkai)
  • cucumber -t @wado (This would only run the top scenarios for Wado)
You get the idea, right?
So there you go, 10 steps to cukey heaven. Enjoy…

Bridging the Gap – Automation

A few months ago I was mulling over the best way to approach a proposed automation initiative I had proposed when I came across a well written and highly informative blog by Elisabeth Hendrickson that helped me focus on my objectives and narrow my search criteria. Our situation is largely similar to a lot of other organisations in that we have a web-based product with a plethora of checks that are ripe for automation. We’re understaffed and overworked (aren’t we all?), so my primary aims included:

  • A way of executing automated functional checks on existing features quickly for each candidate build
  • Transparency across platforms and browsers (Windows, MacOS, Firefox, Chrome, IE, etc.)
  • Allow the sapient testers to focus on testing and discovering new information about the product
  • An efficient way of communicating automation results to all stakeholders without overwhelming them with data
  • A way of involving the non-technical stakeholders so they feel part of the technical side of things (without having to be technical)
  • Providing a way of capturing screenshots that allow manual inspection of aesthetics (also useful for demos, presentations, bug reports, etc)
  • Build some bridges between technology-business, hopefully stimulating further collaboration through visibility

Implementation

I’m a big fan of scripted languages and tend to go for Perl more often than not, but Ruby was catching my eye for this particular adventure. I love the ability to simply  “write’n’run” code without having to deal with compilation, linking, etc or having to invest in expensive IDEs to bring projects to life. Ruby has elegant syntax and a wealth of support in the online community, plus the inclusion of IRB (Interactive Ruby Shell) which was marvellously easy to use when experimenting with new code.

I just wanted an intelligent editor and a command line. As far as editors go, vim  was always an option but as I was going to implement this on a Mac, I plumped for BBEdit which, as Bare Bones Software suggests, “it doesn’t suck”. This afforded me the ability to switch between multiple files very quickly (you can do this with vim too, but I liked the presentation and the reviews were favourable).

Drivers

While the implementation was a no-brainer, the choice of driver – i.e. the thing that would manipulate my web interface was swayed somewhat by my previous experiences with Selenium. Like most people, I had used the Firefox plugin and played with the IDE record/playback functionality and found that the export utility to Ruby was less than perfect. Selenium RC (as it was called at the time) wasn’t much better (for me at least) and I encountered some real problems when trying to interact with javascript elements. It’s much improved now by all accounts but I was already searching for alternatives (a bias on my behalf, I guess).

The other obvious choice was Watir which seemed like a natural fit to me, but after reading a series of posts from Alister Scott, I chose to go with Watir-Webdriver instead. I particularly enjoyed his ‘meaty example‘ post, comparing Ruby automated APIs. The syntax looked elegant as well as affording me the chance to make the code maintainable (as well as readable). In principle, this choice would support IE, Firefox and Chrome across Windows and MacOS (with the possibility of Opera too). It also allowed me to capture ‘full screen’ screenshots (as opposed to ‘visible area’ screenshots) – something which my previous endeavours with Selenium caused more pain by not working as I’d hoped.

Frameworks

There are a number of frameworks to choose from but the first couple I looked at were a bit too ‘codey’ for my non-technical stakeholders, namely Test::Unit and RSpec. I even bought a copy of The Rspec Book but in doing so ended up reading and digesting the chapters within that talked about Cucumber. A soon as I saw this I knew it would give me the best chance of drawing in the non-technical stakeholders. I ended up buying The Cucumber Book as well, but not until I was most of the way through implementation – thankfully there are lots of great online references, tutorials, and examples to learn from on the web. The key things about Cucumber that won me over were:

  • Plain text descriptions
  • Nifty results presentation with the default Cucumber HTML formatter
  • Even niftier results presentation with a customised Cucumber HTML formatter
  • Ability slice and dice the automation suite using Cucumber tags – a very powerful feature

Now, granted we don’t do any sort of BDD at our place, but that didn’t stop me from choosing Cucumber (nor should it, IMHO)

Move to the current day, the initiative is complete and has been well received so far.

I’ll elaborate more on the details in later blog posts.

Mind Mapping Websites

I’ve been working on a different mind map blog for a while but thought I’d post this one first as it’s quick and relatively simple.

There was an initiative at work recently to introduce multi language support for a particular website. Part of the preparation here involved finding some way of documenting the website structure and the text within. A number of people tried to go about this in different ways and encountered different problems until I decided to do it with mind maps, and I’ll share this technique below. It’s not rocket science, granted, but I found mind maps to be a natural foil for something like this.

One of the other more obvious methods I witnessed involved taking screenshots for each page which compiled a hefty screen pack. A fair attempt as it makes sense to keep the reference as close as possible to what you’ll be looking at, but when I started asking questions about response & error messages and conditionally visible elements within web pages it quickly became clear that a single screenshot (or even two, three, four five, etc.) would become a fairly cumbersome method of doing this, not only to compile in the first instance, but also to maintain going forward.

As a tester I wanted a flexible way of documenting this information that would allow me to easily illustrate coverage for each page that was easily understood, and more importantly would be comparatively painless to update going forward – enter mind maps. I looked for common elements in the web pages and came up with the following legend:

Mind Map Format

Granted, our web pages are not that complicated and this structure would naturally have to be re-thought and adapted for more complex sites. In the context for which it was created though, it was a fairly decent fit.
Rather than just leave you with this legend and mosey off into the sunset, I thought it would be prudent to give an example.

Enter my ‘Geeza Fiver’ 1-page website.

A fictitious site that takes a flat £5 donation from your credit card and gives it to one of three well deserving (but non-existent) Scottish charities. (Note – for those of you not familiar with Scottish slang ‘Geeza’ is a short way of saying ‘Give Us A’ or ‘Give Me A’ – one of many odd Scottish colloquialisms is to refer to yourself in the plural (but I digress) and ‘Fiver’ simply means ‘Five Pounds’)

GeezaFiver

Now, I know you’re all reaching for your credit card at this very moment, but before you donate your hard earned cash, here’s how I would have broken this page down using mind maps:

GeezaFiver-Map

You can also supplement this with your own notes to remind yourself how to prompt particular error messages. I tend to use purple italic text as it’s quickly distinguisable from the rest of the map but for the sake of clarity I’ve left such comments out of the above example. Here are a few pros and cons (I’m sure there are more):

Pros
– Simple format. Not overly complicated
– Easily maintainable going forward
– Doesn’t take up oodles of disk space
– Use of colour quickly allows the eye to distinguish response/error text from page text

Cons
– If you’re looking at this before you’ve seen the actual screen, it may not mean as much. It could be used in conjunction with screenshots however for new starters unfamiliar with the product.

As I’m sure you’ve seen over the blogosphere, mind maps can be used for any number of things, but this was one area I found them particularly useful.