BDD Steps: Free your mind!

I just started to implement the steps for something like the following story:

It started to get complicate as I was in need of a parameter converter (JBehave) which converts “Article” into an object representing the document type. After some time I realized that the story was again written by someone who knows the product in too much detail.

I stopped work on the converter and rewrote the story – and now the story tells what the customer really wants:

where “rendered property” (we are editing web-contents here) is another learning to specify cleanly what you want – here: If you want to show the text in the preview later on you first should ensure to choose a property which will become a visible part of the web page.

And suddenly there was no need for a parameter converter anylonger.

I don’t vote against parameter converters – I just have to repeat to myself again and again: Write what you (as customer) really want! – and don’t try to incorporate the knowledge about the test environment (here: document type of type Article exists) into the story.

Doing so will keep your tests more robust over time (perhaps one day there is no “Article” anylonger) and sometimes (as it was here) it will even ease your test code.

For how to handle the references in the stories above you might want to read my blog post “BDD Design Patterns: Reference Tracker”.

BDD Design Patterns: Reference Tracker

Writing BDD stories and implementing the steps you will soon come to the point to deal with references between steps, like:

Given I have a document A on disk
When I open document A
And I change the content of document A
And I save document A
Then document A on disk will be updated
And document A will contain the new content

Reading this you will find two references to be tracked:

• the document itself
• the content which got changed

I found many implemented solutions for this pattern, each of them a little different. Some using global variables and a central class to hold all steps, others using singletons.

Summarizing the solutions I found I introduced a new pattern which got quite positive feedback: the Reference Tracker.

The basic idea is that you will have three items to track:

1. the type of the reference (here: document or content)
2. the reference name (here: A once, unnamed the other)
3. the actual reference value

Having this you just need two methods to feed the tracker and to retrieve values from it:

The cleanup() method should ensure that you don’t collide with new scenarios. In JBehave context you should call it in @AfterScenario.

For the example above you would implement the interface with:

• Key-Type: String (“document”, “content”)
• Value-Type: Object (once file, once String, XML or whatever)

The complete Gist contains another implementation called TypedReferenceTracker which assumes that the value-type is the same as the key, so that there is a class for documents and one for contents. It makes the getter a little easier to use.

Find the complete Gist at GitHub.

Code Kata with DDT in JUnit

Quote from Wikipedia

Code Kata is a term coined by Dave Thomas, co-author of the book The Pragmatic Programmer, in a bow to the Japanese concept of kata in the martial arts. A code kata is an exercise in programming which helps a programmer hone their skills through practice and repetition. As of October 2011, Dave Thomas has published 21 different katas.

Today I participated in my first Code Kata initiated by one of our Scrum Masters. My intention was to get to know this concept in more detail and to see if it is a forum to train BDD skills.

Looking back I am sure that a BDD Code Kata needs some preparation but I also got more practice on Data Driven Testing (DDT). We decided to code the kata Roman Numerals.

We started very simple minded with a test, testing that 1 is converted to I, then 2 to II and of course, simple minded as we are we implemented it by concatenating as many I’s as the number has. And of course this implementation first failed for 4.

First we started writing one test method for each new number we wanted to test. The first discussion which came up:

Shouldn’t we just write one test method testing all numbers?

I answer: No – because if you have 10 assertions and the second one fails you will have no hint about the state of the subsequent 8 assertions as JUnit will just stop this very test. Having multiple short test methods has the advantages:

• Methods are much easier to read.
• The different failed tests might help to isolate the source of the problem. Let’s say a test for all even number fails than you might have a first hint what to look for.

DDT – what?

Data Driven Testing is to have a set of data to feed a test with. They might come from a file or any other data generator. The allurement having data driven tests is that the code to test might be very short but you can do many boundary tests with nearly no additional costs.

The following is an example how to do DDT with JUnit. It’s the result of the short work we did today and of course far from being perfect or complete:

Conclusion

I will take part in the next Code Kata next week. It was very refreshing to be able to train techniques I use rarely (like DDT) and to discuss several approaches to a problem in a group.

Links

• Image from morguefile.com
• Image edited with Aviary Advanced Image Editor
• Software Testing Glossary
• Coding Dojo Wiki
• it-agile about Code-Katas and Coding-Dojos (German)

BDD Steps: Be specifically unspecific

I just participated in an interesting sprint retrospective meeting where the problem came up that step definitions are sometimes very ambiguous as well as for the human reader as for the JBehave parser.
Here is one typical though virtual example:

Problems of the steps above:

• The Given statement contains information with no value for the customer. The customer is not interested about the username and the password. Thus even with this start it will be hard to understand the complete scenario.
• The first When statement misses important information. Is this a valid or invalid login? When implementing this step developers might as side effect create a new user with these credentials. The next one re-using this step in a story might not know about this side effect.
• The second When statement collides with the first one if you parse parameters. If you are unlucky you even might have two step definitions in your code:
  
  with the effect that it is up to JBehave if the user logs in as "named qwert" or "qwert" depending on which step it has chosen to take.

What we can learn from it is to be specifically unspecific... or to be more precise, apply these rules to your stories:

1. Give every step a value (if not even every single phrase).
   In the first statement it might be better to state either that I logged in as normal user or that I logged in as a specific user I want to reference later in the test.
2. Never specify field values explicitly.
   Actually this refers to the first rule as most of the time the real content of a field is not important. It might be important of what type the input is (alphabetical, numeric, valid login) but most of the time the real input is not important.
3. Be as specific as needed and as unspecific as possible.
   Perhaps the rule with the most problems to decide what is the best approach. It means that you should not be too specific for example by stating "When I fill the username field with username qwert" as perhaps some day your username field disappears and is replaced by a fingerprint scanner. But you should state if the login is valid or invalid and if it is invalid: if it is because the user doesn't exist or because the password is wrong.
   

So the following steps will be easier to understand... even without the context of prior or subsequent steps. And they never state any field value:

Then vs. Period–Writing JBehave Stories

Writing stories/scenarios which get automatically executed as in JBehave is sometimes a special art – at least if you want to re-use existing step definitions. Sticking to some rules will help to make your steps re-usable, one is for example to agree on the tense to use in your stories as I mentioned earlier.

I stumbled across another rule when I was writing steps and then implementing them. Here an example:

Implementing the @Then step might look like this:

Referring to the title of this post you might guess the problem: For some yet unknown reason the first scenario marked the @Then step as pending. Solution after hours of debugging (just kidding): The period at the end of the sentence is missing in the step definition!

So you could add an @Alias for this (no, really, you don't want that). Next time you will stumble across parameterized steps where the parameter is at the end of the sentence. Suddenly the parameter will get a period at the end. And suddenly usernames for example don’t match.

The solution is quite simple. I found it in some existing stories of a team:

Don’t use periods!

Really, I hate this approach as from my point of view every new rule will prevent stories to be written. But I think this rule helps so much, that you should stick to it.

Are Testers that special?

Recently I stumbled across a very interesting job post. So to say, it was the best one I have ever read. It was from XING searching for a test engineer. It did not contain any you must now that and not less statements but a sentence like that: You know [...] and you learn what you don't know yet. Yes, that's exactly the point - if you are good, you can always learn.

The most interesting part was this statement:

• You see  the error in this sentence.

Do you see it? I saw it immediately. Now I ask myself, is it really a hint for skilled testers that they see the error?

Given, When, Then - Which Tense?

If you do impromptu writing of BDD steps in your scenarios you will experience a broad variety of wordings for the same thing. Sometimes its just because you switched between present tense, past tense or future.

Having a best practice at hand for choosing the right tense will at least reduce the number of varieties. My recommendation is:

@Given - Past Tense

@Given describes a precondition. Something that already should have happened. Thus my recommendation is to use Past Tense here.

@When - Present Tense

@When describes an action you do just right now. I think Present Tense matches here very well.

@Then - Future

@Then describes an expected state you will reach. I think Future fits here best.

What do you think?

And what about you? Do you have any standards here?

Behavior Driven What? BDA Waiting...

This is the start of a small series about BDA - Behavior Driven Antipatterns. I am currently reading through a bunch of JBehave stories at my company to give tips how to optimize them. While doing so I got some insights on antipatterns in writing these stories and I would like to share them with you. My first issue of this series is about Waiting. Waiting is especially an issue when testing AJAX applications.

First lets have a look of an example close to one I found at some locations:

Why do we have to wait anyway? Because we have to deal with asynchronous events - especially when dealing with Web User Interfaces. If we press a button there will be an unpredictable roundtrip until we get the response which we want to validate.

Why is the given approach an antipattern? Because stories (at first glance) should be stories written by customers or their representatives like the product owner (Scrum). And guess what? Customers don't want to wait. Never! So they would never state anything about waiting. at first glance is a lookahead on an idea I am developing about different flavors of stories. I hope to come to that later in this series.

So what are the alternatives? Giving a reason... If you have many places in your story files which look like this I recommend to extend the step a little:

What's the advantage? When you added the pause to your tests you knew the reason why you did that. When other developers read the same line sometimes later they won't. So giving a hint why you decided to wait will help others to understand your thoughts and perhaps speed up the tests because they reduce the time to wait or they find some better solution. Will come to that one later.

Alternative: Hide waiting... This solution comes closer to what the customer really wants which is not to wait. Just remove the wait step. Now you need to add some Thread.sleep() to your step definition instead. You just have to decide if to add it to the @Given or @Then step. To decide keep in mind if you will reuse the step and if perhaps in other contexts waiting will just slow down the test for no reason.

Recommended Alternative: Busy waiting... The best alternative to any sleep approach is to do busy waiting. I would recommend to add busy waiting to the step which requires a precondition before it can start. This would be the @Then step in the given scenario. To do busy waiting there are several alternatives out there and many implementations in your company, I bet. But if you are already using Selenium and/or WebDriver you might want to use one of these examples:

Both examples use the same trick which fits the basic principle of busy waiting which is:

• Check if condition is met.
• If it is met stop waiting.
• If it isn't met wait a little and try again.
• If the condition isn't met within a given total time (timeout) abort with a failure.

The advantages are:

• The step clearly fails with a precondition not met if there is a problem. Otherwise the step will fail later on with no real hint what happened.
• If the precondition is met fast the test will continue fast.
• Your tests are more robust because you may even raise the time to wait without many headaches.
• The story is cleaned from any waiting issues.

May I Introduce Myself?

This is the first post in my new blog. While my other Blogs are insights to my private life and the great fun I have with my two children this one is about my professional experiences as Software Architect/Engineer Quality Assurance.
A short summary of my CV:

• I studied computer science with focus on Neural Computation.
• I finished my Diplom thesis in 2000 navigating a robot through a maze based on information derived from optical flow.
• Since 2000 I worked mainly as Software Engineer and experienced my passion for quality assurance testers.
• In 2005 I changed to CoreMedia AG, Hamburg, now working there as Software Architect Quality Assurance. I started right away with GUI-Test-Automation with HP WinRunner (before: Mercury Interactive), switched over to QF-Test and nowadays collect experiences with Selenium to drive Web-UI-Tests.
• Currently I am collecting some experiences in doing behavior driven testing based on JBehave.

Well, that's all about me for now. You can also meet me at:

• Xing
• Linkedin
• Twitter
• Facebook