A friend recently told me that as many as 80% of the apps sold in apps store aren't tested; even by their programmers. I couldn't find this statistic anywhere so I have no idea if it is really true but it got me thinking: Could this be true?
I've been a QA (quality assurance engineer) in the software industry for a while now. I started as a programmer and migrated to QA. You could say I understand the programmer's point of view. Many apps are written by individuals or very small teams. Deadlines in the software world can be tyranny. Programmers often do test their work. As a matter of fact, in almost every case where I have found a bug in code the programmer has at some point said, "It worked for me..."
But what is quality when we are talking about a mobile application? Why does it matter? How do you know when it's "in there"?
Everybody will tell you they want quality. If an app freezes, crashes, is slow or awkward to use we bemoan the poor quality. "Who tested this piece of junk?!" we protest. To understand why things don't work (or don't work well) we have to go back to the very beginning and look at our assumptions. They are our assumptions because they influence everything we buy. And our assumptions color our understanding of quality.
Myth 1. Testing Improves Quality. The truth: Testing does not improve quality.
This is the first and most fundamental truth in all of Quality Assurance. Testing does not improve quality, testing reveals quality or lack of it. Don't believe me? Ask yourself. When has taking a classroom quiz ever made you smarter? Never. It may, however, reveal your strengths and weaknesses. Likewise more testing does not improve anything unless it is done within a process which takes what you have learned and applies fixes and improvements.
Myth 2. Quality is the result of rigorous testing. The truth: Quality is not the result of rigorous testing.
Even when you have a process to learn from your mistakes quality is not the result of testing. Why not? Because that would simply take too long. Testing, even if it begins before the end of the process, is too late to have the type of impact on quality that we are looking for.
For a designer quality begins with the very first glimmer of what they want their app to accomplish.
With every wish on the designer's wish list there is a list of questions from a QA engineer:
How will we prove this? (Is it true?)
How will we measure this? (How true is this?)
How will we assure this? (How will we make this stay true for the millionth user?)
It is only by asking the right questions that an app can approach being a quality solution.
Why does anyone care about quality?
Because in the end quality is the only true measure of value. I am always surprised how, for every app that cost 99 cents, there are five-star raves and one-star pans. Why would two sets of people have completely different experiences from the same app?
One answer might be that the programmers did the testing themselves. Ironic? Think back (for some of you this may be way back) to the last time you did poorly on a test. Two things were probably true. In fact I dare say if they both weren't true you would have done better on the test.
You didn't know the material as well as you thought you did and...
You did not write the test yourself!
You really cannot test yourself just like I cannot proof read my own blog. The programmer/author is too close to the work to objectively scrutinize their work. Semafores knows this. That is why they have a quality assurance person on the founding team. Someone to help assure the right questions are asked so that quality is not an afterthought but the result of good engineering.
Quality. The result of all the right preparation.