Archive for February, 2010

Po Po Po Po POWER Testing

February 18, 2010

One our first employees is called Franklyn. He is from the Dominican Republic and started his testing carreer a bit more then 5 years ago with TestABil. Franklyn has been an boxer in his youth and also has a pretty good hit for baseball. He still is a baseball fanatic.

Over the years in various projects in TestABil he has created a very good sense for defects. He can smell them from a distance. Is he the most organised tester, no. Is he a automation crack, no. Franky is getting bored very quick if he has not shouted DEFECTAZO at least every 2 hours. He moves in his chair, clicks on back and forward buttons, refreshes the screen… he will get really nervous. He needs to put all his power on his test to push the bugs to light. In our Mantis defect metrics he is the absolute number one.

A short time ago we started a testing project for a new online database. In a few weeks time my team was very busy with writing and executing test cases. But… the number of defects was a bit low. The customer (at 800 km distance) was wondering why there were so few defects.

I explained that we were doing a full test and that we had a lot of test cases to do. But, as the customer wanted to know quickly if the provider had delivered a ´bug´ free product we had to do something. I proposed the customer that we could do a one day ´bug hunt´ without executing “formal”  testcases. They agreed as they quickly wanted to see some bugs.

I called in Franklyn for one day (he is in another project) to help.

The result:

  • totally 78 bugs reported (70% severity major&crash), with step by step descriptions.
  • We identified new possible test cases
  • The team learned some test techniques in practice
  • The customer happy because we could warn them on time.

The bug winner: Franklyn with 38 (among which 70% severity major & crash). So we heard more then 25 times “DEFECTAZO”  in one day!

The rest of the team was impressed and also learned some tricks from him.

They now call him the PO-PO-PO-PO POWER TESTER.

What kind of test is this: exploratory, well no, although we explored a lot we were to focussed on bugs, also it is done when you are already doing structured testing, we really could confirm that the reported bugs were bugs. Is it a bug hunt, well maybe, although I like the following term: POWER TESTING, a one day test execution using powerful test techniques to bring to light high severity bugs.

I believe this test is ideal for web sites that already are on the web. Or projects that have decreasing defect metrics. Just to see…what  you cannot see…

Power Testing

Advertisements

Organise your test – the bug tracker

February 17, 2010

The bug tracker is for me the most important tool in a Test Project.  When I start a new project the first thing I put in place is the bug/issued/defect tracker.  The issue tracker is the central place of a project. When you start to reviewing the project and you have doubts or issues. Raise a issue in the bug tracker. When you make the plan raise an issue.

In that way you easily log everything and reduce emails (you loose), numurous meetings without meeting minutes etc.

In my opinion the best tool is a web based tool that can be configured easily and is user friendly. I have used excel, Access, Traq, Jira, Bugzilla, Mantis and HQ Quality Center. But for me the number 1 is Mantis. It is so easy to use and to configure, has a nice user interface and has good reports. It is also a tool that can be implemented very easy.

I recommend to use as much as possible the standard configuration of the tool. It easy to add additional fields, but before doing that evaluate if those additional fields really add any value. Keep it simple.

When reporting try to make write clear summaries. Both for project related issues and for bugs/defects.

For example (bug summary): When logging in with an incorrect user name the error message is unclear.

Or for an (project) issue: The funtional documentation is not delivered on the agreed date.

It is important to set some clear rules, a defect/issue process, that need to be followed by the whole team. The main rule for me: Reporter of issues is responsible for closing the issue. When I say responsible it does not mean he has to close it him/herself, but at least be aware of who is closing the defect.

For more info on my favourite open-source defect tracker: http://www.mantisbt.org/