A competitive usability study was employed to measure user performance and user preference for immersive virtual environments (VEs) with multimodal gestural interfaces when compared directly with nonstereoscopic traditional CAD interfaces that use keyboard and mouse. The immersive interfaces included a wand and a data glove with voice interface with an 86 in. stereoscopic rendering to screen; whereas, the traditional CAD interfaces included a 19 in. workstation with keyboard and mouse and an 86 in. nonstereoscopic display with keyboard and mouse. The context for this study was a set of “real world” engineering design scenarios. These design scenarios include benchmark 1—navigation, benchmark 2—error finding and repair, and benchmark 3—spatial awareness. For this study, two populations of users were employed, novice (n = 15) and experienced (n = 15). All users experienced three successive trials to quantify the effects of limited learning. Statistically based comparisons were made using both parametric and nonparametric methods. Conclusions included improved capability and user preference for immersive VEs and their interfaces were statistically significant for navigation and error finding/repair for immersive interfaces.
Issue Section:
Research Papers
References
1.
Johnson
, M. W.
, 2002
, “Boeing Phantom Works—Personal Communication on VR Design
,” Dr. George Hazelrigg of the National Science Foundation, Arlington, VA.2.
Nielsen
, J.
, 1993
, Usability Engineering
, 1st ed., Academic Press
, San Diego, CA
.3.
Gould
, J. D.
, and Lewis
, C.
, 1985
, “Designing for Usability: Key Principles and What Designers Think
,” Commun. ACM
, 28
(3
), pp. 300
–311
.10.1145/3166.31704.
ISO (International Standards Organization)
, 2003
, Guidance on Usability
, ISO
, Geneva, Switzerland
.5.
Hix
, D.
, Swan
, J. E.
, Gabbard
, J.
, McGee
, M.
, Durban
, J.
, and King
, T.
, 1999
, “User-Centered Design and Evaluation of a Real-Time Battlefield Visualization Virtual Environment
,” IEEE
Virtual Reality
, Houston, TX, Mar. 13–17, pp. 96
–103
.10.1109/VR.1999.7569396.
Gabbard
, J.
, Hix
, D.
, and Swan
, J. E.
, 1999
, “User Centered Design and Evaluation of Virtual Environments
,” IEEE Comput. Graphics Appl.
, 19
(6
), pp. 51
–59
.10.1109/38.7997407.
Gabbard
, J.
, Swartz
, K.
, Ritchey
, K.
, and Hix
, D.
, 1999
, “Usability Evaluation Techniques: A Novel Method for Assessing the Usability of an Immersive Medical Visualization VE
,” Visual Worlds and Simulation (VWSIM'99)
, San Francsico, CA, pp. 165–170.8.
Nielsen
, J.
, 1993
, “Iterative User Interfaces Design
,” IEEE Computer
, 26
(11
), pp. 32
–41
.10.1109/2.2414249.
Poupyrev
, I.
, Weghorst
, S.
, Billinghurst
, M.
, and Ichikawa
, T.
, 1997
, “A Framework and Testbed for Studying Manipulation Techniques for Immersive VR
,” ACM
Conference on Virtual Reality Systems and Technology
, Lausanne, Switzerland, pp. 21
–28
.10.1145/261135.26114110.
Stanney
, K. M.
, Mollaghasemi
, M.
, Reaves
, L.
, Breaux
, R.
, and Graeber
, D. A.
, 2003
, “Usability Engineering of Virtual Environments (VEs): Identifying Multiple Criteria That Drive Effective VE System Design
,” Int. J. Hum. Comput. Stud.
, 58
(4
), pp. 447
–481
.10.1016/S1071-5819(03)00015-611.
Kotranza
, A.
, Lok
, B.
, Deladisma
, A.
, Carla
, M.
, and Lind
, D. S.
, 2009
, “Mixed Reality Humans: Evaluating Behavior, Usability, and Acceptability
,” IEEE Trans. Visualization Comput. Graphics
, 15
(3
), pp. 369
–382
.10.1109/TVCG.2008.19512.
Gill
, S. A.
, and Ruddle
, R. A.
, 1998
, “Using Virtual Humans to Solve Real Ergonomic Design Problems
,” International Conference on Simulation
, York, UK, pp. 223
–229
.10.1049/cp:1998064113.
Gupta
, R.
, Whitney
, D.
, and Zeltzer
, D.
, 1997
, “Prototyping and Design for Assembly Analysis Using Multimodal Virtual Environments
,” Comput. Aided Des.
, 29
(8
), pp. 585
–598
.10.1016/S0010-4485(96)00093-014.
Sreedharan
, S.
, Zurita
, E. S.
, and Plimmer
, B.
, 2007
, “3D Input for 3D Worlds
,”, 19th Australasian Conference on Computer-Human Interaction: Entertaining User Interfaces. OZCHI '07
, pp. 227–230.10.1145/1324892.132494015.
Simon
, A.
, 2005
, “First Person Experience and Usability of Co-Located Interaction in a Projection-Based Virtual Environment
,” ACM
Symposium on Virtual Reality Software and Technology
, pp. 23
–30
.10.1145/1101616.110162216.
Patel
, H.
, Stefani
, O.
, Sharples
, S.
, Hoffman
, H.
, Kraseitanidis
, I.
, and Amditis
, A.
, 2006
, “Human Centered Design of 3-D Interaction Devices to Control Virtual Environments
,” Int. J. Hum. Comput. Stud.
, 64
(3
), pp. 207
–220
.10.1016/j.ijhcs.2005.08.01017.
Banerjee
, P.
, and Basu-Mallick
, D.
, 2003
, “Measuring the Effectiveness of Presence and Immersive Tendencies on the Conceptual Design Review Process
,” ASME J. Comput. Inf. Sci. Eng.
, 3
(2
), pp. 166
–179
.10.1115/1.157850018.
Johansson
, P.
, and Yennerman
, A.
, “Immersive Visual Interfaces-Assessing Usability by the Effects of Learning/Results From an Empirical Study
,” ASME J. Comput. Inf. Sci. Eng.
, 4
(2
), pp. 124
–131
.10.1115/1.174077519.
Thomas
, P.
, and Macredie
, R. D.
, 2002
, “Introduction to the New Usability
,” ACM Trans. Comput. Hum. Interaction
, 9
(2
), pp. 69
–73
.10.1145/513665.51366620.
Chu
, C.-C.
, Dani
, T.
, and Gadh
, R.
, 1998
, “Evaluation of Virtual Reality Interface for Product Shape Designs
,” IIE Trans.
, 30
(7
), pp. 629
–643
.21.
Herndon
, K.
, Van Dam
, A.
, and Gleicher
, M.
, 1994
, “The Challenges of 3D Interaction
,” Conference on Human Factors in Computing Systems
, Boston, MA, pp. 36
–43
.22.
Kaur
, K.
, Sutcliffe
, A.
, and Maiden
, N.
, 1999
, A Design Advise Tool Presenting Usability Guidance for Virtual Environments
, British HCI Group, King's Manor, University of York
, York, UK
.23.
Computer Science Department—Communications Research Group, University of Nottingham,
1997
, “COVEN—Collaborative Virtual Environments—Usage Evaluation
,” Public Delivrable Report of ACTS, Project No. AC042.24.
Nomura
, J.
, and Sawada
, K.
, 1999
, “Virtual Reality Technology and Its Applications to Industrial Use
,” Control Eng. Pract.
, 7
(11
), pp. 1381
–1394
.10.1016/S0967-0661(99)00114-825.
Ware
, C.
, Arthur
, K.
, and Booth
, K.
, 1993
, “Fish Tank Virtual Reality
,” Conference on Human Factors in Computing Systems
, Amsterdam, The Netherlands, pp. 37
–42
.10.1145/169059.16906626.
Bolt
, R.
, and Herranz
, E.
, 1992
, “Two-Handed Gesture in Multi-Modal Natural Dialog
,” ACM
Symposium on User Interface, Software, and Technology
, Monterey, CA, pp. 7
–18
.10.1145/142621.14262327.
Wexelblat
, A.
, 1995
, “An Approach to Natural Gesture in Virtual Environments
,” ACM Trans. Comput. Hum. Interaction
, 2
(3
), pp. 179
–200
.10.1145/210079.21008028.
Hand
, C.
, 1997
, “A Survey of 3D Interaction Techniques
,” Comput. Graphics Forum
, 16
(5
), pp. 269
–281
.10.1111/1467-8659.0019429.
Stork
, A.
, and Maidhof
, M.
, 1997
, “Efficient and Precise Solid Modeling Using a 3D Input Device
,” ACM
Conference on Solid Modeling
, Atlanta, GA, pp. 181
–194
.10.1145/267734.26778230.
Trika
, S. N.
, Bannerjee
, P.
, and Kashyap
, R. L.
, 1997
, “Virtual Reality Interfaces for Feature-Based Computer-Aided Design Systems
,” Comput. Aided Des.
, 29
(8
), pp. 565
–574
.10.1016/S0010-4485(96)00092-931.
Tushar
, D.
, and Gadh
, R.
, 1997
, “Creation of Concept Shape Designs Via a Virtual Reality Interface
,” Comput. Aided Des.
, 29
(8
), pp. 555
–563
.10.1016/S0010-4485(96)00091-732.
Jayaram
, S.
, Wang
, Y.
, Jayaram
, U.
, Lyons
, K.
, and Hart
, P.
, 1999
, “Virtual Assembly Design Environment
,” IEEE Virtual Reality Annual International Symposium
, Los Alamitos, CA, pp. 172
–179
.33.
Jayaram
, S.
, Connacher
, H.
, and Lyons
, K.
, 1997
, “Virtual Assembly Using Virtual Reality Techniques
,” Comput. Aided Des.
, 29
(8
), pp. 575
–584
.10.1016/S0010-4485(96)00094-234.
Lu
, S. C.-Y.
, Shpitalni
, M.
, and Gadh
, R.
, 1999
, “Virtual and Augmented Reality Technologies for Product Realization
,” CIRP Ann. Manuf. Technol.
, 48
(2
), pp. 471
–495
.10.1016/S0007-8506(07)63229-635.
Bowman
, D.
, Johnson
, D.
, and Hodges
, L.
, 1999
, “Testbed Evaluation of Virtual Environment Interaction Techniques
,” ACM
Conference on Virtual Reality Systems and Technology
, London, UK, pp. 26
–33
.10.1145/323663.32366736.
Evans
, P.
, Vance
, J.
, and Dark
, V.
, 1999
, “Assessing the Effectiveness of Traditional and Virtual Reality Interfaces in Spherical Mechanism Design
,” ASME J. Mech. Des.
, 121
(4
), pp. 507
–514
.10.1115/1.282949037.
Antonya
, C.
, and Talaba
, D.
, 2007
, “Design Evaluation and Modification of Mechanical Systems in Virtual Environments
,” Virtual Reality
, 11
(4
), pp. 275
–285
.10.1007/s10055-007-0074-638.
Zhang
, R.
, Oliver
, J. H.
, Noon
, C.
, Gilmore
, B.
, Winer
, E.
, and Duncan
, J.
, 2007
, “Immersive Product Configurator for Conceptual Design
,” ASME
Paper No. DETC2007, pp. 1403
–1413
. 10.1115/DETC2007-3539039.
Zhang
, R.
, Noon
, C.
, Winer
, E.
, Oliver
, J.
, Gilmore
, B.
, and Duncan
, J.
, 2007
, “Immersive Product Configurator for Conceptual Design
,” AIAA/ASME/ASCE/AHS/ASC
Structures, Structural Dynamics and Materials Conference, pp. 2839
–2848
.10.2514/6.2007-193140.
LaViola
, J.
, 1999
, “Whole-Hand and Speech Input in Virtual Environments
,” M.S. thesis, Computer Science, Brown University, Providence, RI.41.
Rosenfield
, R.
, Zhu
, X.
, Toth
, A.
, Shriver
, S.
, Lenzo
, K.
, and Black
, A.
, 2000
, “Towards a Universal Speech Interface
,” International Conference on Spoken Language Processing Beijing
, China.42.
Dorojkin
, D. V.
, and Vance
, J. M.
, 2002
, “Implementing Speech Recognition in Virtual Reality
,” ASME
Paper No. DETC2002/CIE-34390.10.1115/DETC2002/CIE-3439043.
Kiyokawa
, K.
, Takemura
, H.
, and Yokoya
, N.
, 2000
, “SeamlessDesign for 3D Object Creation
,” IEEE MultiMedia
, 7
(1
), pp. 22
–33
.10.1109/93.83930844.
Kuester
, F.
, Duchaineau
, M.
, Hamann
, B.
, Joy
, K.
, and Ma
, K.-L.
, 2000
, “DesignersWorkbench: Towards Real-Time Immersive Modeling
,” SPIE, Stereoscopic Displays and Virtual Reality Systems
, San Jose, CA, pp. 464
–472
.45.
Omata
, M.
, Go
, K.
, and Imamiya
, A.
, 2000
, “A Gesture-Based Interface for Seamless Communication Between Real and Virtual Worlds
,” Sixth Ercim Workshop “User Interfaces For All,” European Consortium for Informatics and Mathematics
, Florence, Italy.46.
Broll
, W.
, and Schardt
, T.
, 2000
, “The Virtual Round Table—A Collaborative Augmented Multi-User Environment
,” ACM Proceedings of the Third International Conference on Collaborative Virtual Environments
, pp. 39
–45
.47.
Volkov
, S.
, and Vance
, J. M.
, 2001
, “Effectiveness and Haptic Sensation for the Evaluation of Virtual Prototypes
,” ASME J. Comput. Inf. Sci. Eng.
, 1
(2
), pp. 123
–128
.10.1115/1.138456648.
Ansar
, A.
, Rodrigues
, D.
, Desai
, J. P.
, Daniilidis
, K.
, Kumar
, V.
, and Campos
, M. F. M.
, 2001
, “Visual and Haptic Collaborative Tele-Presence
,” Comput. Graphics
, 25
(5
), pp. 789
–798
.10.1016/S0097-8493(01)00121-249.
50.
Campbell
, D. A.
, and Wells
, M.
, 2003
, A Critique of Virtual Reality in the Architectural Design Process
, Human Interface Technology Laboratory, University of Washington
, Seattle, WA
.51.
Chipperfield
, K. A.
, and Vance
, J.
, 2003
, “VRHose—Virtual Hydraulic Hose Routing Design
,” Material Handling Division
, American Society of Mechanical Engineering
, pp. 57
–63
.52.
Jayaram
, S.
, Jayaram
, U.
, Gadh
, R.
, Srinivasan
, H.
, and Vance
, J.
, 2001
, “Assessment of VR Technology and Its Applications to Engineering Problems
,” ASME J. Comput. Inf. Sci. Eng.
, 1
(1
), pp. 72
–83
.10.1115/1.135384653.
Gaisemier
, J.
, von Bohuszewicz
, O.
, Gehnen
, G.
, and Grafe
, M.
, 1996
, “Integration of Real IT-Applications in Virtual Manufacturing Systems
,” ASME Design Engineering Technical Conferences and Computers in Engineering Conference
, Irvine, CA, pp. 1
–10
.54.
Demiralp
, C.
, Jackson
, C.
, Karelitz
, D.
, Zhang
, S.
, and Laidlaw
, D. H.
, 2006
, “CAVE and Fishtank Virtual-Reality Displays: A Qualitative and Quantitative Comparison
,” IEEE Trans. Visualization Comput. Graphics
, 12
(3
), pp. 323
–330
.10.1109/TVCG.2006.4255.
Lin
, J. J.-W.
, Duh
, H. B. L.
, Abi-Rached
, H.
, Parker
, D. E.
, and Abi-Rached
, H.
, and Furness
, T. A.
, 2002
, “Effects of Field of View on Presence, Enjoyment, Memory, and Simulator Sickness in a Virtual Environment
,” IEEE
Virtual Reality Conference
, IEEE Computer Society
, Orlando, FL
, pp. 164
–171
.10.1109/VR.2002.99651956.
Duh
, H. B.-L.
, Lin
, J. J. W.
, Furness
, T. A.
, Kenyon
, R. V.
, and Parker
, D. E.
, 2001
, “Effects of Field of View on Balance in an Immersive Environment
,” IEEE Virtual Reality Conference
, IEEE
Computer Society, Yokohama, Japan, Mar. 17, pp. 235
–240
.10.1109/VR.2001.91379157.
Tan
, D. S.
, Gergle
, D.
, Scupelli
, P.
, and Pausch
, R.
, 2003
, “With Similar Visual Angles, Larger Displays Improve Spatial Performance
,” SIGCHI
Conference on Human Factors in Computing Systems
, ACM, Ft. Lauderdale, FL, pp. 217
–224
.10.1145/642647.64265058.
Tyndiuk
, F.
, Lespinet-Najib
, V.
, Thomas
, G.
, and Schlick
, C.
, 2004
, “Impact of Large Displays on Virtual Reality Task Performance
,” Computer Graphics, Virtual Reality, Visualisation and Interaction in Africa Conference
(AFRIGRAPH '04), ACM, Stellenbosch, South Africa, pp. 61
–65
.10.1145/1029949.102996059.
Hinckley
, K.
, Tullio
, J.
, Pausch
, R.
, Profitt
, D.
, and Kassell
, N.
, 1997
, “Usability Analysis of 3D Rotation Techniques
,” ACM
Symposium on User Interface Software and Technology
, pp. 1
–10
.10.1145/263407.26340860.
Schulze
, J. P.
, Forsberg
, A. S.
, Kleppe
, A.
, Zeleznik
, R. C.
, and Laidlaw
, D. H.
, 2005
, “Characterizing the Effect of Level of Immersion on a 3D Marking Task
,” HCI International Conference
, Las Vegas, NV.61.
Pausch
, R.
, Proffitt
, D.
, and Williams
, G.
, 1997
, “Quantifying Immersion in Virtual Reality
,” ACM SIGGRAPH Conference on Computer Graphics
, pp. 13
–18
.10.1145/258734.25874462.
Yeh
, Y.
, and Silverstein
, L.
, 1992
, “Spatial Judgments With Monoscopic and Stereoscopic Presentation of Perspective Displays
,” Hum. Factors
, 34
(5
), pp. 583
–600
.63.
Arthur
, K.
, Booth
, K.
, and Ware
, C.
, 1993
, “Evaluating 3D Task Performance for Fish Tank Virtual Worlds
,” ACM Trans. Inf. Syst.
, 11
(3
), pp. 239
–265
.10.1145/159161.15535964.
Ware
, C.
, and Balakrishnan
, R.
, 1994
, “Reaching for Objects in VR Displays: Lag and Frame Rate
,” ACM Trans. Comput. Hum. Interaction
, 1
(4
), pp. 331
–356
.10.1145/198425.19842665.
Barfield
, W.
, Hendrix
, C.
, and Bystrom
, K.
, 1997
, “Visualizing the Structure of Virtual Objects Using Head Tracked Stereoscopic Displays
,” Virtual Reality Annual International Symposium
, Albuquerque, NM, Mar. 1–5, pp. 114
–120
.10.1109/VRAIS.1997.58305266.
Chance
, S.
, Gaunet
, F.
, Beall
, A.
, and Loomis
, J.
, 1998
, “Locomotion Mode Affects the Updating of Objects Encountered During Travel
,” Presence
, 7
(2
), pp. 168
–178
.10.1162/10547469856565967.
Watson
, B.
, Spaulding
, V.
, Walker
, N.
, and Ribarsky
, R.
, 1997
, “Evaluation of the Effects of Frame Time Variation on VR Task Performance
,” Virtual Reality Annual International Symposium
, Albuquerque, NM, Mar. 1–5, pp. 38
–44
.10.1109/VRAIS.1997.58304268.
Narayan
, M.
, Waugh
, L.
, Zhang
, X.
, Bafna
, P.
, and Bowman
, D.
, 2006
, “Quantifying the Benefits of Immersion for Collaboration in Virtual Environments
,” ACM
Symposium on Virtual Reality Software and Technology
, Monterey, CA, pp. 78
–81
.10.1145/1101616.110163269.
Polys
, N.
, Kim
, S.
, and Bowman
, D.
, 2005
, “Effects of Information Layout, Screen Size, and Field of View on User Performance in Information-Rich Virtual Environments
,” ACM
Symposium on Virtual Reality Software and Technology
, Monterey, CA, pp. 46
–55
.10.1145/1101616.110162670.
Polys
, N.
, Kim
, S.
, and Bowman
, D.
, 2006
, “Effects of Information Layout, Screen Size, and Field of View on User Performance in Information Rich Virtual Environments
,” Comput. Animation Virtual Worlds
, 18
(1
), pp. 19
–38
.10.1002/cav.15971.
Arns
, L.
, Cook
, D.
, and Cruz-Neira
, C.
, 1999
, “The Benefits of Statistical Visualization in an Immersive Environment
,” IEEE
Virtual Reality Annual International Symposium
, Houston, TX, Mar. 13–17, pp. 88
–95
.10.1109/VR.1999.75693872.
Gruchalla
, K.
, 2004
, “Immersive Well-Path Editing: Investigating the Added Value of Immersion
,” IEEE Virtual Reality
, Mar. 27–31, pp. 157
–164
.10.1109/VR.2004.131006973.
Swan
, J.
, Gabbard
, J.
, Hix
, D.
, Schulman
, R.
, and Kim
, K.
, 2005
, “A Comparative Study of User Performance in a Map-Based Virtual Environment
,” IEEE
Virtual Reality Conference
, Los Angeles, CA, pp. 259
–266
.10.1109/VR.2003.119114974.
Schuchardt
, P.
, and Bowman
, D.
, 2007
, “The Benefits of Immersion for Spatial Understanding of Complex Underground Cave Systems
,” ACM
Virtual Reality Software and Technology
, Newport Beach, CA, Mar. 22–26, pp. 121
–124
.10.1145/1315184.131520575.
Prabhat
, Forsberg
, A.
, Katzourin
, M.
, Wharton
, K.
, and Slater
, M.
, 2007
, “A Comparative Study of Desktop, Fishtank and Cave Systems for the Exploration of Volume Rendered Confocal Data Sets
,” IEEE Trans. Visualization Comput. Graphics
, 13
(3
), pp. 551
–563
.10.1109/TVCG.2007.7043376.
Chung
, J.
, 1992
, “A Comparison of Head-Tracked and Non-Head-Tracked Steering Modes in the Targeting of Radiotherapy Treatment Beams
,” ACM
Symposium on Interactive 3D Graphics
, pp. 193
–196
.10.1145/147156.14720477.
Bliss
, J.
, Tidwell
, P.
, and Guest
, M.
, 1997
, “The Effectiveness of Virtual Reality for Administering Spatial Navigation Training to Firefighters
,” Presence
, 6
(1
), pp. 73
–86
.78.
Nielsen
, J.
, and Phillips
, V.
, 1993
, “Estimating the Relative Usability of Two Interfaces: Heuristic, Formal, and Empirical Methods Compared
,” SIGCHI
Conference on Human Factors in Computing Systems
, Amsterdam, The Netherlands, pp. 214
–221
.10.1145/169059.16917379.
GraphPad Software,
2003
, “InStat
,” San Diego, CA.80.
SAS Institute
, 1999
, “SAS for Windows
,” 00M0 ed., SAS Institute, Cary, NC.81.
Satter
, K. M.
, 2005
, “Competitive Usability Studies of Virtual Environments for Shipbuilding
,” Ph.D. dissertation, Engineering Management, University of New Orleans, New Orleans, LA.Copyright © 2015 by ASME
You do not currently have access to this content.
