The Benefits of Recreating Physical Products and Spaces in the Lab

The office was abuzz; there was chatter of testing a futuristic shower. Hearing this news, we (Blink researchers) lifted our heads from our screens like prairie dogs emerging from their dens after a long winter. We were intrigued. Working at Blink UX we see a variety of products that need testing and/or evaluation. We often perform tests on digital devices such as mobile phones, tablets, and computers. However, there are times when we need to observe user behavior that extends beyond a screen – and this always excites us!

In the past 17 years of conducting research we have seen a lot of interesting projects. Many that stick out in our memory are the ones that involve full-scale prototypes. These are projects where observing the user interacting with the product in environment is necessary to understand behaviors, implications, and usability. We have conducted studies with full-scale prototypes including the aforementioned interactive shower, a digital faucet, an operating room, multiple living room lay outs, and an electrical box that required participants to use a ladder to access it, to name a few. When we build a full-scale prototype we transform our lab space into an interactive and scaled version of the product and environment where participants are able to experience and illustrate common user scenarios.

Many projects that stick out in our memory are the ones that involve full-scale prototypes.

Most recently we did this for a client needing to test an interface that involved a gas pump. We immediately recognized that this was an excellent opportunity to use a full-scale prototype. What made the decision so apparent? There are many reasons for using this strategy but we will focus on what we consider the most influential: cost, logistics, and testing conceptual products.

Cost

Cost is a large factor when deciding whether to create a full-scale prototype or use an actual device/environment. Often times it is more cost effective to recreate the setting or device. For example, if we reserved an active gas pump at a store location for a week solely for testing, thousands of dollars would be lost. Specifically, we calculated the cost of testing at a location with a steady flow of cars where one pump serves 12 cars an hour. An average car has a 12-gallon gas tank and let’s say that these cars get unleaded gas at $2.559 a gallon. With these numbers it would cost the business approximately $14,739.84 to reserve one gas pump for a week. In addition to the cost of a researcher, another team member would be needed to help out with logistics and organization adding more overhead expense. By testing in a lab we remove this loss for our client. For instance when we created a pump and tested in the lab we drastically reduced the cost (including time and materials).

When considering this method of testing, researchers must consider how data collection and interactions are affected using mock devices/environments. If the difference is negligible and this strategy is able to answer key research questions while understanding interactions, then you know it is worthwhile.

Scalability

Logistics and scalability are additional arguments for making full-scale models. Organizing and designing a study in a lab eases recruiting, participant coordination, data collection, and is not disruptive to the client’s business. When recruiting and coordinating in a lab there is a process in place where there is a designated meeting place and location for participants to wait, meet with the moderator, and sign consent forms. When recruiting participants in the field an adapted system would be needed, where there is a “wrangler” to welcome and organize the participants. In addition, a waiting area must be created while the participants wait for their turn. If you take the gas pump research as an example, you can imagine it would be a bit awkward for participants to sit inside a gas station while waiting to participate.

Another logistical advantage of conducting the study in-lab is having a controlled environment. In our lab we are able to collect data and capture key interactions with high-quality recording equipment. When we tested the mock gas pump we set up multiple camera shots where pivotal exchanges took place. We had a feed focused on a phone screen, the pump console screen, and overview shot where we could observe how participants interacted with the mock pump, mock car, and phone. By having this set up we captured the challenges of managing multiple devices while pumping gas.

In addition, having a controlled lab study eliminates distractions and mitigates unforeseen events. Based on our experience with field intercept studies there are multiple sources for distraction. For example, when we conduct tests in retail stores we often experience intermittent PA announcements, lounge music playing the background, and interruptions by curious onlookers. You can imagine that testing a gas pump in the field would yield the same disruptions, with the potential for more. The lab provides an environment where the researcher is able to control the pace of the session and where the participant is focused and easy to hear.

In our lab we are able to collect data and capture key interactions with high-quality recording equipment.

An instance where opting for a lab environment was extremely helpful was when we evaluated workflows, tools, and interactions in an operating room. By doing this we avoided disrupting hospital operations, provider and patient schedules. We successfully recreated key scenarios in our labs and where we observed and talked to targeted individuals such as nurses. With this body of research we helped create a system and tool that was efficient, relevant, and valuable for end users that worked in operating rooms.

Prototyping

Sometimes we build full-scale prototypes when testing product concepts. There are instances where we are asked to evaluate technologies that do not exist. Specifically, we are tasked with assessing the validity of a conceptual product along with understanding potential user behavior and interactions. Cognitive walkthroughs are not always enough to get honest, helpful reactions in these situations. The scenario must be experienced first hand to gauge interactions, usability issues, and the value of a product. To uncover deep user insights sometimes it is necessary to perform key tasks in a similar environment and/or with a device. To accomplish this, we create a prototype that meets design and interaction criteria.

A great example of this type of product testing is the work we did with Moen on the U by Moen shower. Moen approached us to test out a new concept for an IoT shower experience, which had never been done before. For usability sessions we created a mock shower in the lab to evaluate how participants reacted and used a digital device to control shower temperature, time, and starting experience. This provided insightful data on how participants interacted and moved in a shower space, where to place such a device in a shower, expectations on how the system should work, and a direction for the product. Moen recently released this product into the market where it received rave reviews.

When setting out to test devices that are not in your typical wheelhouse such as gas pumps, showers, and operating rooms create a full-scale mock prototype to test in the lab. This method significantly reduces cost, eases planning, and allows accurate data to be collected for conceptual products. We know from experience that using this strategy aids in the development and direction of a product while providing useful user insights and feedback.

Darshana Tuladhar, when not observing and analyzing human behavior and interactions, enjoys birds, playing the violin, drinking root beer, and salsa dancing, preferably simultaneously.