Testing software
A student who graduated last year and now working for a software company came to see me. He said: “I work as software tester, I test everything very carefully but my customer still finds defects. What did I do wrong and what could I do to be a better tester?”
I told him: “If you have done your best then maybe you did not do anything wrong. A good tester should know that it is impossible to test everything. Testing does not remove all defects and the fact is some of these defects may happen because of other factors. It is possible that the requirements given to you may be incomplete or they are still changing when you conduct your test. The ability to test software depends on the skills of the testers to extract the right information to build test cases. If you do not have all the information or if that information changes often then you may not be able to build good test cases. So it may not be your entire fault.
He asked: “How can I make sure that I do NOT have any defects?”
I told him:” It is impossible to test everything so knowing what could possibly be tested is a good start. Testing knowledge at any given point in a project is determined by what could be tested at that time. If you still have defect, instead of create more test case to ensure the defect doesn’t occur again, you must evaluate “Why” it happened. What information were you missing, and why were you missing it? Instead of evaluating the test coverage you may need to look at your testing techniques. Evaluate what and how you are testing throughout the project, and you may ask others in your project to help you evaluate what you are doing.
He told me: “My manager always complained: “Why didn’t you find this defect before it was released?” and I am worry that I may not be able to keep my job”.
I told him: “It is easy to blame tester for software problem. Many people believe that the purpose of testing is to “remove all defects” in software. The reality is testing can only prove the existing of defects but never prove the absence of defects. A good manager should know that testers do not work with customer to understand their needs, testers do not write requirements, testers do not architect the system, testers do not design the system’s features, and testers do not develop the solution so blaming the tester without knowing where the defects come from is NOT fair. I often wonder why manager do not ask, “Why is this software designed so poorly”, or “why a function is so poorly planned?” Why don’t we ask “what assumptions did the programmer make that lead to injecting problems into the code”, and “what can we do to prevent recurrent errors in the future?” Why senior manager do not ask project managers on why they put the burden of ‘quality’ and the success of a software product SOLELY on testers.
I believe there are two reasons why testers are blamed for software defect. First, people mistakenly assume the objective of testing is ONLY to find and fix defects. Second, testers are held accountable for things they have NO control over because of management belief that lesser experienced or newly graduates should work as testers. We know that it is impossible to test everything, and if the tester is inexperienced, do not receive adequate training then the potential to miss defects will increases significantly.
In my opinion, the objective of software testing is to provide “Information about the attributes and capabilities of the software under test” to people responsible for making critical business decisions. Because these information will allow them to understand the potential risks in relation to their expectations of the software prior to release. If the defect number is high, they may order more tests or allow more time for testing to reduce the chance of having defects after release to customer. Software defects provide the decision makers and other team members with information about the software so project manager may order “root cause analysis” or more software reviews and inspections to identify WHERE the defect come from and fix it so it will NOT happen again; It is important to “Build quality into software” instead of “Testing for quality” because testing will never prove the absence of defects, there is no such thing as zero defects as it only exist in theories. The major problem is project managers do not give adequate time for other activities so they do not have time to do good jobs. Poorly done jobs will create defects and problems then they blame on software testers.
To improve software and achieve customer satisfaction, we have to look at the entire process of software development from the beginning to the end to identify where problems happen. We can not only look at the end then blame testers.
Sources
- Blogs of Prof. John Vu, Carnegie Mellon University
