Assembly process of complex electromechanical products can be quite complicated and time consuming because of high quality demands. Aiming at improving the efficiency of the manual assembly process, this paper proposes an automatic interaction method using part recognition for augmented reality (AR) assembly guidance, which improves both the accuracy of part picking and the interaction efficiency of AR guidance system. Taking sample images of similar parts as input and part types as output, a deep neural network model Part R-CNN for part recognition is build based on Faster R-CNN and is further fine-tuned by back propagation. By recognizing the assembly part, the augmented assembly guidance information of the corresponding parts assembly process is triggered in real-time without direct user interaction. Experimental results show that the deep neural network based part recognition method reaches 94% on mean average precision and the average recognition speed is 200ms per image frame. The average speed of AR guidance content triggering is about 20fps. All system performance satisfies the accuracy and real-time requirements of the AR-aided assembly system.
Skip Nav Destination
ASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
August 26–29, 2018
Quebec City, Quebec, Canada
Conference Sponsors:
- Design Engineering Division
- Computers and Information in Engineering Division
ISBN:
978-0-7918-5173-9
PROCEEDINGS PAPER
An Automatic Interaction Method Using Part Recognition Based on Deep Network for Augmented Reality Assembly Guidance
Xuyue Yin,
Xuyue Yin
Shanghai Jiao Tong University, Shanghai, China
Search for other works by this author on:
Xiumin Fan,
Xiumin Fan
Shanghai Jiao Tong University, Shanghai, China
Search for other works by this author on:
Jiajie Wang,
Jiajie Wang
Shanghai Jiao Tong University, Shanghai, China
Search for other works by this author on:
Rui Liu,
Rui Liu
Shanghai Jiao Tong University, Shanghai, China
Search for other works by this author on:
Qiang Wang
Qiang Wang
Shanghai Jiao Tong University, Shanghai, China
Search for other works by this author on:
Xuyue Yin
Shanghai Jiao Tong University, Shanghai, China
Xiumin Fan
Shanghai Jiao Tong University, Shanghai, China
Jiajie Wang
Shanghai Jiao Tong University, Shanghai, China
Rui Liu
Shanghai Jiao Tong University, Shanghai, China
Qiang Wang
Shanghai Jiao Tong University, Shanghai, China
Paper No:
DETC2018-85810, V01BT02A018; 10 pages
Published Online:
November 2, 2018
Citation
Yin, X, Fan, X, Wang, J, Liu, R, & Wang, Q. "An Automatic Interaction Method Using Part Recognition Based on Deep Network for Augmented Reality Assembly Guidance." Proceedings of the ASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. Volume 1B: 38th Computers and Information in Engineering Conference. Quebec City, Quebec, Canada. August 26–29, 2018. V01BT02A018. ASME. https://doi.org/10.1115/DETC2018-85810
Download citation file:
82
Views
Related Proceedings Papers
Related Articles
Case Studies on Modeling Manufacturing Processes Using Artificial Neural Networks
J. Eng. Ind (August,1995)
Manufacturing Assembly Time Estimation Using Structural Complexity Metric Trained Artificial Neural Networks
J. Comput. Inf. Sci. Eng (March,2014)
Neural Network Models for Usage Based Remaining Life Computation
J. Eng. Gas Turbines Power (January,2008)
Related Chapters
Prediction of Coal Mine Gas Concentration Based on Constructive Neural Network
International Conference on Information Technology and Computer Science, 3rd (ITCS 2011)
Estimating Resilient Modulus Using Neural Network Models
Intelligent Engineering Systems Through Artificial Neural Networks, Volume 17
Comparison of Bayesian Estimation and Neural Network Model in Stock Market Trading
Intelligent Engineering Systems through Artificial Neural Networks Volume 18