Abstract
The human performance benefits of four types of guided assembly instructions, including the Microsoft HoloLens 1, were analyzed in the context of a realistic assembly task. Several studies have confirmed the benefits of using augmented reality (AR) work instructions over traditional digital or paper instructions, but few have compared the effects of different AR hardware, including head-mounted displays, for complex assembly tasks. Participants completed a mock wing assembly task using the Microsoft HoloLens 1, and completion time, error count, and net promoter score (NPS) were recorded. These data were compared to data from previous research which employed desktop model-based instructions (MBIs), tablet MBI, and tablet AR instructions for the same task. The use of HoloLens AR instructions led to time saving of 16% over the tablet AR instructions. HoloLens users also had lower error rates than non-AR users. Despite the performance benefits of the HoloLens AR instructions, this condition had a lower NPS than the tablet AR group. The qualitative data showed that some users thought the HoloLens device was uncomfortable and that the tracking was not always exact. Although the users favored the tablet AR condition, the HoloLens condition had significantly faster assembly times. The authors recommend using the HoloLens for complex guided assembly instructions with minor changes, such as allowing the user to toggle the AR instructions on and off at will. The results of this paper can help manufacturing stakeholders understand the benefits of different AR technologies for manual assembly tasks.
Issue Section:
Research Papers
References
1.
Liao
, T.
, and Humphreys
, L.
, 2015
, “Layar-Ed Places: Using Mobile Augmented Reality to Tactically Reengage, Reproduce, and Reappropriate Public Space
,” New Media Soc.
, 17
(9
), pp. 1418
–1435
. 10.1177/14614448145277342.
Boeing
, 2018
, “Boeing Tests Augmented Reality in the Factory
,” Boeing [Online], http://www.boeing.com/features/2018/01/augmented-reality-01-18.page, Accessed February 28, 2018.3.
Palmarini
, R.
, Erkoyuncu
, J. A.
, Roy
, R.
, and Torabmostaedi
, H.
, 2018
, “A Systematic Review of Augmented Reality Applications in Maintenance
,” Rob. Comput. Integr. Manuf.
, 49
, pp. 215
–228
. 10.1016/j.rcim.2017.06.0024.
Caudell
, T. P.
, and Mizell
, D. W.
, 1992
, “Augmented Reality: An Application of Heads-Up Display Technology to
,” Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences
, Kauai, HI
, Jan. 7–10
, Vol. 2
, IEEE
, pp. 659
–669
.5.
Wiedenmaier
, S.
, Oehme
, O.
, Schmidt
, L.
, and Luczak
, H.
, 2003
, “Augmented Reality (AR) for Assembly Processes Design and Experimental Evaluation
,” Int. J. Hum. Comput. Interact.
, 16
(3
), pp. 497
–514
. 10.1207/S15327590IJHC1603_76.
Echtler
, F.
, Sturm
, F.
, Kindermann
, K.
, Klinker
, G.
, Stilla
, J.
, Trilk
, J.
, and Najafi
, H.
, 2004
, “The Intelligent Welding Gun: Augmented Reality for Experimental Vehicle Construction,” Virtual and Augmented Reality Applications in Manufacturing
, S. K.
Ong
, and A. Y. C.
Nee
, eds., Springer London
, London, UK
, pp. 333
–360
.7.
Nilsson
, S.
, and Johansson
, B.
, 2007
, “Fun and Usable
,” Proceedings of the 2007 Conference of the Computer–Human Interaction Special Interest Group (CHISIG) of Australia on Computer–Human Interaction: Design: Activities, Artifacts and Environments—OZCHI ‘07
, Adelaide, Australia
, Nov. 28–30
, ACM Press
, New York
, p. 123
.8.
Henderson
, S.
, and Feiner
, S.
, 2011
, “Augmented Reality in the Psychomotor Phase of a Procedural Task
,” 2011 10th IEEE International Symposium on Mixed and Augmented Reality
, Basel, Switzerland
, Oct. 26–29
, IEEE
, pp. 191
–200
.9.
Tang
, A.
, Owen
, C.
, Biocca
, F.
, and Mou
, W.
, 2003
, “Comparative Effectiveness of Augmented Reality in Object Assembly
,” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
, Fort Lauderdale, FL
, Apr. 5–10
, pp. 73
–80
.10.
Richardson
, T.
, Gilbert
, S.
, Holub
, J.
, Macallister
, A.
, Radkowski
, R.
, Davies
, P.
, and Terry
, S.
, 2014
, “Fusing Self-Reported and Sensor Data From Mixed-Reality Training
,” Interservice/Industry Training, Simulation and Education Conference (I/ITSEC)
, Orlando, FL
, Dec. 1–4
, pp. 1
–12
, Paper No. 14158
.11.
Tatić
, D.
, and Tešić
, B.
, 2017
, “The Application of Augmented Reality Technologies for the Improvement of Occupational Safety in an Industrial Environment
,” Comput. Ind.
, 85
, pp. 1
–10
. 10.1016/j.compind.2016.11.00412.
Azuma
, R.
, 1997
, “A Survey of Augmented Reality
,” Presence
, 6
(4
), pp. 355
–385
.13.
Azuma
, R.
, Baillot
, Y.
, and Behringer
, R.
, 2001
, “Recent Advances in Augmented Reality
,” IEEE Comput. Graph. Appl.
, 21
(6
), pp. 34
–47
.14.
Boud
, A.
, Haniff
, D.
, Baber
, C.
, and Steiner
, S.
, 1999
, “Virtual Reality and Augmented Reality as a Training Tool for Assembly Tasks
,” IEEE International Conference Information Visualization
, London, UK
, July 14–16
, pp. 32
–36
.15.
Henderson
, S.
, and Feiner
, S.
, 2009
, “Evaluating the Benefits of Augmented Reality for Task Localization in Maintenance of an Armored Personnel Carrier Turret
,” 8th IEEE International Symposium on Mixed and Augmented Reality, 2009
, Orlando, FL
, Oct. 19–22
.16.
Hou
, L.
, Wang
, X.
, and Truijens
, M.
, 2015
, “Using Augmented Reality to Facilitate Piping Assembly: An Experiment-Based Evaluation
,” J. Comput. Civil Eng.
, 29
(1
), p. 05014007
. 10.1061/(ASCE)CP.1943-5487.000034417.
Sanna
, A.
, Manuri
, F.
, Lamberti
, F.
, Member
, S.
, Paravati
, G.
, and Pezzolla
, P.
, 2015
, “Using Handheld Devices to Support Augmented Reality-Based Maintenance and Assembly Tasks
,” IEEE International Conference on Consumer Electronics Using
, Las Vegas, NV
, Jan. 9–12
, pp. 178
–179
.18.
Nee
, A. Y. C.
, Ong
, S. K.
, Chryssolouris
, G.
, and Mourtzis
, D.
, 2012
, “Augmented Reality Applications in Design and Manufacturing
,” CIRP Ann.
, 61
(2
), pp. 657
–679
. 10.1016/j.cirp.2012.05.01019.
Biocca
, F. A.
, and Rolland
, J. P.
, 1998
, “Virtual Eyes Can Rearrange Your Body: Adaptation to Visual Displacement in See-Through, Head-Mounted Displays
,” Presence
, 7
(3
), pp. 262
–277
. 10.1162/10547469856570320.
Cuervo
, E.
, 2017
, “Beyond Reality
,” GetMobile Mob. Comput. Commun.
, 21
(2
), pp. 9
–15
. 10.1145/3131214.313121821.
Feiner
, S.
, Macintyre
, B.
, and Seligmann
, D.
, 1993
, “Knowledge-Based Augmented Reality
,” Commun. ACM
, 36
(7
), pp. 53
–62
. 10.1145/159544.15958722.
De Crescenzio
, F.
, Fantini
, M.
, Persiani
, F.
, Di Stefano
, L.
, Azzari
, P.
, and Salti
, S.
, 2011
, “Augmented Reality for Aircraft Maintenance Training and Operations Support
,” IEEE Comput. Graph. Appl.
, 31
(1
), pp. 96
–101
. 10.1109/MCG.2011.423.
Microsoft
, 2018
, “Why HoloLens,” [Online], https://www.microsoft.com/en-us/hololens/why-hololens, Accessed February 26, 2018.24.
Loch
, F.
, Quint
, F.
, and Brishtel
, I.
, 2016
, “Comparing Video and Augmented Reality Assistance in Manual Assembly
,” Proceedings—12th International Conference on Intelligent Environments, IE 2016
, London, UK
, Sept. 14–16
, pp. 147
–150
.25.
Doshi
, A.
, Smith
, R. T.
, Thomas
, B. H.
, and Bouras
, C.
, 2017
, “Use of Projector Based Augmented Reality to Improve Manual Spot-Welding Precision and Accuracy for Automotive Manufacturing
,” Int. J. Adv. Manuf. Technol.
, 89
(5–8
), p. 1288
. 10.1007/s00170-016-9164-526.
Uva
, A. E.
, Gattullo
, M.
, Manghisi
, V. M.
, Spagnulo
, D.
, Cascella
, G. L.
, and Fiorentino
, M.
, 2017
, “Evaluating the Effectiveness of Spatial Augmented Reality in Smart Manufacturing: A Solution for Manual Working Stations
,” Int. J. Adv. Manuf. Technol.
, 94
(1–4
), pp. 509
–521
. 10.1007/s00170-017-0846-427.
Zheng
, X. S.
, Foucault
, C.
, Matos
, P.
, Silva
, D.
, Dasari
, S.
, Yang
, T.
, and Goose
, S.
, 2015
, “Eye-Wearable Technology for Machine Maintenance: Effects of Display Position and Hands-Free Operation
,” Proceedings of the 33rd Annual ACM Conference Human Factors in Computing Systems
, Seoul, South Korea
, Apr. 18–23
, pp. 2125
–2134
.28.
Baird
, K. M.
, and Barfield
, W.
, 1999
, “Evaluating the Effectiveness of Augmented Reality Displays for a Manual Assembly Task
,” Virtual Real.
, 4
(4
), pp. 250
–259
. 10.1007/BF0142180829.
Wickens
, C.
, Gordon
, S.
, Liu
, Y.
, and Lee
, J.
, 1998
, An Introduction to Human Factors Engineering
, Pearson Education, Inc.
, New Jersey
.30.
Neumann
, U.
, and Majoros
, A.
, 1998
, “Cognitive, Performance, and Systems Issues for Augmented Reality Applications in Manufacturing and Maintenance
,” Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No. 98CB36180)
, Atlanta, GA
, Mar. 14–18
, pp. 4
–11
.31.
Funk
, M.
, Kosch
, T.
, and Schmidt
, A.
, 2016
, “Interactive Worker Assistance: Comparing the Effects of In-Situ Projection, Head-Mounted Displays, Tablet, and Paper Instructions
,” Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing
, Heidelburg, Germany
, Sept. 12–16
, ACM, New York, pp. 934
–939
.32.
Funk
, M.
, Kosch
, T.
, Greenwald
, S. W.
, and Schmidt
, A.
, 2015
, “A Benchmark for Interactive Augmented Reality Instructions for Assembly Tasks
,” Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia—MUM ‘15
, Linz, Austria
, Nov. 30–Dec. 2
, ACM Press, New York, pp. 253
–257
.33.
Funk
, M.
, Mayer
, S.
, Nistor
, M.
, and Schmidt
, A.
, 2016
, “Mobile In-Situ Pick-by-Vision
,” Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments—PETRA ‘16
, Corfu, Greece
, June 29–July 1
, ACM Press, New York, pp. 1
–4
.34.
Blattgerste
, J.
, Strenge
, B.
, Renner
, P.
, Pfeiffer
, T.
, and Essig
, K.
, 2017
, “Comparing Conventional and Augmented Reality Instructions for Manual Assembly Tasks
,” Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments—PETRA ‘17
, Island of Rhodes, Greece
, June 21–23
, pp. 75
–82
.35.
Syberfeldt
, A.
, Danielsson
, O.
, Holm
, M.
, and Wang
, L.
, 2015
, “Visual Assembling Guidance Using Augmented Reality
,” Procedia Manuf.
, 1
, pp. 98
–109
. 10.1016/j.promfg.2015.09.06836.
Kantowitz
, B. H.
, 1987
, “Mental Workload,” Human Factors Psychology
, P. A.
Hancock
, ed., North-Holland
, Amsterdam, Netherlands
, pp. 81
–121
.37.
MacAllister
, A.
, Hoover
, M.
, Gilbert
, S.
, Oliver
, J.
, Radkowski
, R.
, Garrett
, T.
, Holub
, J.
, Winer
, E.
, Terry
, S.
, and Davies
, P.
, 2017
, “Comparing Visual Assembly Aids for Augmented Reality Work Instructions
,” Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC)
, Orlando, FL
, Nov. 27–Dec. 1
.38.
Microsoft
, “Install the Tools—Mixed Reality|Microsoft Docs
” [Online], https://docs.microsoft.com/en-us/windows/mixed-reality/install-the-tools#mixed-reality-toolkit, Accessed April 4, 2018.39.
Träskbäack
, M.
, and Haller
, M.
, 2004
, “Mixed Reality Training Application for an Oil Refinery
,” Proceedings of the 2004 ACM SIGGRAPH International Conference on Virtual Reality Continuum and Its Applications in Industry—VRCAI ‘04
, Singapore
, June 16–18
, ACM Press
, New York
, p. 324
.40.
Radkowski
, R.
, Herrema
, J.
, and Oliver
, J.
, 2015
, “Augmented Reality-Based Manual Assembly Support With Visual Features for Different Degrees of Difficulty
,” Int. J. Hum. Comput. Interact.
, 31
(5
), pp. 337
–349
. 10.1080/10447318.2014.99419441.
Schwerdtfeger
, B.
, and Klinker
, G.
, 2008
, “Supporting Order Picking With Augmented Reality
,” 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality
, Cambridge, UK
, Sept. 15–18
, IEEE, pp. 91
–94
.42.
Miller
, J.
, Hoover
, M.
, and Winer
, E.
, 2019
, “Overcoming Limitations of the HoloLens For Use in Product Assembly
,” IS&T International Symposium on Electronic Imaging Science and Technology.
, San Francisco, CA
, Jan. 13–17
.43.
Ekstrom
, R.
, French
, J.
, Harman
, H.
, and Dermen
, D.
, 1976
, Manual for Kit of Factor Referenced Cognitive Tests
, Educational Testing Service
, Princeton, NJ
.44.
Macallister
, A.
, Gilbert
, S.
, Holub
, J.
, Winer
, E.
, and Davies
, P.
, 2016
, “Comparison of Navigation Methods in Augmented Reality Guided Assembly
,” Interservice/Industry Training, Simulation, Education Conference (I/ITSEC)
, Orlando, FL
, Nov. 28–Dec. 2
, pp. 1
–14
.45.
Hoover
, M.
, MacAllister
, A.
, Holub
, J.
, Gilbert
, S.
, Winer
, E.
, and Davies
, P.
, 2016
, “Assembly Training Using Commodity Physiological Sensors
,” Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC).
, Orlando, FL
, Nov. 28–Dec. 2
.46.
Reichheld
, F.
, 2003
, “The One Number You Need to Grow
,” Harv. Bus. Rev.
, 81
(12
), pp. 46
–54
.47.
Creswell
, J.
, and Creswell
, D.
, 2017
, Research Design: Qualitative, Quantitative, and Mixed Methods Approaches
, Sage
, Los Angeles, CA
.Copyright © 2020 by ASME
You do not currently have access to this content.
