Perhaps you’ve dealt many times with people who have wrong assumptions about testing. They used to come from a lack of experience using metrics and testing techniques. Metrics are the essential measures we need to collect after making decisions. That’s the difference between designing based on data instead of intuition.
Here is some advice that you can use to separate the fact from fiction and dispel the myths. Use them as a weapon to convince those who don’t want to include testing in the design process.
1. Metrics takes too much time
I’m sure that almost everyone has thought or heard someone with this statement.
The only thing that takes too much time it’s to re-built some functionality that didn’t work.
Metrics will speed up the design process and will solve doubts and wrong assumptions. Besides, with these measures, you will be able to defend your design with arguments and will clear any discussion about it.
Some quick solutions are e-mailing or surveying, that can take just a few extra hours or minutes of your time.
2. Testing and UX Metrics cost too much money
One of the most popular myths about testing is that it is too much expensive.
But, have I mentioned e-mailing or surveying?
Among others, these are two essential tools to get valuable data frequently and diagnose usability issues.
Also, some of the analysis tools are actually free on the web and you can install for free click and eye trackers or heat-maps tools.
3. Metrics have nothing to do with new products
………. Hey minions, what do you think about it ?
Otherwise, you won’t be able to compare any information in the future. You should set the target metrics you want to measure in function of your product nature. Some general metrics could be satisfaction, efficiency, efficacy and task success.
4. Small sample size doesn’t provide reliable data
There is a widely belief that you need amounts of users to collect trustworthy data, but the best things often come in small packages.
Although having a larger set of users and data will help you in your decisions, you can have smaller samples of 5, 10 or 20 participants that can increase your confidence level. In fact, there is a camp that claims:
With 5 participants you will discover the 80% of usability issues from your site.
It will depend too on the experience you are measuring. For example, you will need more participants on a survey than in a task test to get a reliable result.
Nielsen & Landauer, 1993.
5. Managers don’t appreciate metrics
Every UX knows that dealing with managers it’s not an easy thing and sometimes they can be rigid or inflexible.
By my own experience, as more as I relate metrics with calculating ROI, the more support I get.
Most managers don’t know about the potential in metrics and the impact they have into the experience, but they understand numbers.
So, you should work in the way you share your metrics and translate them into business language.
For example, avoid making conclusions without numbers to support them. It’s not the same say we have a problem with the forms, that say that the 45% of users spend 1.3 minutes on average fixing form validation errors.
