Baseline experience testing
At Blink we use baseline experience testing to develop qualitative benchmarks for the usability of a product or service offering. Benchmarks identify areas that need design improvements and can later be used to evaluate design changes.
Representative users are given realistic tasks to accomplish while a study moderator observes and identifies usability problems. We find that a baseline experience test, along with a usability inspection such as a heuristic review, is a good way to start improving the overall user experience of a product or service. They provide clear and actionable areas to work on and a means to see if design changes are working. Organizations that are new to UX (generally) and usability (specifically) find them helpful for introducing teams to UX.
We generally recommend sample sizes of 8-14 study participants for baseline experience testing. Though primarily qualitative by design, some quantitative indicators such as task success rates and satisfaction scores are often reported in these studies.
When larger samples are required or statistical confidence is needed, Blink typically uses samples of 15 or more study participants to establish quantitative usability benchmark metrics for a product. These might include time on task, error rates, completion rates, or user satisfaction scores. Benchmark tests can also be replicated to quantify design improvements between development builds or releases, compare performance against competing products, or assess more than one potential design in “A/B” testing.