People are already familiar with 2-D digital experiences. We interact with flat screens every day, ranging from smartphones and desktop computers to smart watches and televisions. Some experiences on flat screens, such as video games and movies, simulate depth, but the experience is fundamentally flat; images are shown on a two-dimensional plane.
For most of us, 3-D digital experiences are newer and more limited. My first 3-D digital experience was watching the 3-D movie “Honey I Shrunk the Audience!” at Disney World. The experience was incredible but was also passive – all I did was sit back and watch. Today, 3-D movies have become more common, but these 3-D experiences do not require any interactions from the viewer. This is about to change.
With the emergence of virtual and augmented reality, we will have many more 3-D digital experiences available to us. Many of these upcoming 3-D experiences will be interactive, requiring the user to navigate a 3-D space. While it may take some time for these experiences to catch on and become pervasive, They will.
3-D software usability study
Recently, we performed a usability study of an augmented reality monitor in the market called zSpace (See Figure 1). Today, thousands of students are already using zSpace technology, which is sold to schools in the form of STEM (Science, Technology, Engineering and Math) labs. When a school purchases a lab, it includes training for teachers, instructional videos and documentation for both teachers and students. In the past we found that training and instructional materials are frequently ignored, so for this study, we evaluated zSpace without training or instructional materials to better understand its out-of-box usability and identify any challenges experienced by first-time users of the product. We found that users’ prior experiences with flat digital spaces influenced how they tried to interact with the 3-D environment and limited what they were able to discover.
Participants in our study interacted with a program called zSpace Experience on the zSpace 200 monitor. They had opportunities to interact with a 3-D chessboard, look inside a beating heart, and take apart a mechanical arm. To interact with these objects in zSpace, participants used a stylus that resembled a laser pointerBy pointing at objects in space and pressing a button they could select them, rotate them, and move them around. For our participants this was their first experience using zSpace, and for some of them it was also their first experience interacting with objects in virtual or augmented reality.
By the end of their research sessions, participants found the experience in zSpace delightful and fun. In particular, they imagined that zSpace could have a significant impact on education. Being able to see and interact with 3-D models helped our participants learn concepts where visualization is essential, such as anatomy, architecture, and engineering. I would have loved having access to this technology in graduate school while I was learning about the functions of the brain. It would have been far easier learning how different brain regions are connected using zSpace than by looking at the flat images in my cognitive neuroscience textbook.
Although our participants enjoyed using zSpace, interacting with objects in zSpace did not come naturally to them. Most of them needed prompting and coaching to fully manipulate objects in the 3-D environment.
Challenge 1: Touching the screen
Upon first opening the zSpace Experience program, participants were presented with a table in a park that contained a chessboard. To select a chess piece, one had to point the stylus at a piece and click a button; there was no need to touch the screen.
When I asked participants to start interacting with the scene in front of them, however, most of the participants brought the stylus to the screen and touched it to the chess piece they wanted to select. Some quickly learned that touching the screen had no effect, but others did not. Two participants went even farther than just touching the screen, they dragged the stylus across the screen to interact with it.
Participants’ initial interactions were guided by their past experiences with digital technology. They had utilized touch screens and styluses before and assumed that the stylus had to touch the screen to affect things in the program.
Challenge 2: Navigating depth
After learning that they didn’t need to touch the screen to manipulate objects, participants still had difficulty navigating the 3-D environment. They had difficulty moving items in the depth dimension and did not fully take advantage of the space that the virtual environment gave them.
One of the most delightful aspects of zSpace is the ability to bring objects in front of the monitor, close to your eyes where they appear to hover right in front of you. When the objects are brought directly in front of your eyes, they look as though they are solid objects that exist outside of the monitor. For example, participants had the opportunity to bring the chess pieces close to their eyes to look at them in detail. Yet most participants did not discover that they could drag objects close towards them without prompting. The objects were in a 3-D space, yet participants acted as if the objects were still in 2-D space.
Challenge 3: Rotating objects
Rotating objects also proved challenging for participants. In zSpace you can rotate selected objects by twisting the stylus in different directions. These twisting motions did not come naturally to participants. They did not anticipate that they could rotate objects, and they did not expect that manipulating a stylus in these ways would have an effect on the program.
Learning how to navigate in 3-D
The findings from our zSpace study show the importance of onboarding and contextual help for navigation in a 3-D environment. The zSpace Experience program did not present our participants with any information about how to interact with the digital environment. Participants had to learn on their own what the buttons on the stylus did and how they could interact with objects in the program. Yet, they wanted the program to present this information to them.
With the lack of instructions in zSpace, it is not surprising that users were initially unable to fully navigate the 3-D space. They were unfamiliar with navigation in a digital 3-D environment and relied upon their past experiences with flat digital interfaces. The behaviors of our participants illustrate that when people approach new technology, their initial understandings of interaction are heavily influenced by their past experiences. These past experiences can help us discover things but also can get in our way when interacting with something that follows very different conventions – such as the jump from 2-D to 3-D interaction.
In the coming years many new augmented reality and virtual reality experiences will hit the market. These experiences will be exciting and novel for users. They will be so novel that users may not discover how to successfully interact with objects in 3-D environments. Onboarding and contextual help could be one way to assist users learn how to successfully interact with new digital paradigms. User research will also be beneficial, as it can help uncover what users are able to discover with or without external assistance. As I wrote in a previous blog post, interactions in virtual reality are essentially experiments that need to be validated through users.
Augmented reality and virtual reality technologies will offer many new, delightful interactions to users – but only to those who are able to discover them. To minimize frustration and make users’ first experiences with 3-D environments successful, well-designed help and onboarding will be essential.