Abstract
When maintenance resources in a manufacturing system are limited, a challenge arises in determining how to allocate these resources among multiple competing maintenance jobs. This work formulates an online prioritization problem to tackle this challenge using a Markov decision process (MDP) to model the system behavior and Monte Carlo tree search (MCTS) to seek optimal maintenance actions in various states of the system. Further, case-based reasoning (CBR) is adopted to retain and reuse search experience gathered from MCTS to reduce the computational effort needed over time and to improve decision-making efficiency. The proposed method results in increased system throughput when compared to existing methods of maintenance prioritization while also reducing the computation time needed to identify optimal maintenance actions as more information is gathered. This is especially beneficial in manufacturing settings where maintenance decisions must be made quickly to minimize the negative performance impact of machine downtime.
References
1.
Chang
, H. S.
, Fu
, M. C.
, Hu
, J.
, and Marcus
, S. I.
, 2005
, “An Adaptive Sampling Algorithm for Solving Markov Decision Processes
,” Oper. Res.
, 53
(1
), pp. 126
–139
. 2.
Albers
, S.
, 2009
, “Online Scheduling
.” Introduction to Scheduling
, Y.
Robert
and F.
Vivien
, eds. CRC Press
, ch. 3.3.
Keizer
, M. C. O.
, Flapper
, S. D. P.
, and Teunter
, R. H.
, 2017
, “Condition-Based Maintenance Policies for Systems With Multiple Dependent Components: A Review
,” Eur. J. Oper. Res.
, 261
(2
), pp. 405
–420
. 4.
Alrabghi
, A.
, and Tiwari
, A.
, 2015
, “State of the Art in Simulation-Based Optimisation for Maintenance Systems
,” Comput. Ind. Eng.
, 82
, pp. 167
–182
. 5.
Chong
, A. K. W.
, Mohammed
, A. H.
, Abdullah
, M. N.
, and Rahman
, M. S. A.
, 2019
, “Maintenance Prioritization–a Review on Factors and Methods
,” J. Facil. Manage.
, 17
(1
), pp. 18
–39
.6.
Saaty
, T. L.
, 2008
, “Decision Making with the Analytic Hierarchy Process
,” Int. J. Serv. Sci.
, 1
(1
), pp. 83
–98
.7.
Sharma
, S.
, and Sisodia
, A.
, 2016
, “Prioritization of Tools in Joint Production-Maintenance Environment of Auto Component Manufacturer Using AHP–Fuzzy–TOPSIS
,” Intell. Ind. Syst.
, 2
(1
), pp. 73
–84
. 8.
Khanlari
, A.
, Mohammadi
, K.
, and Sohrabi
, B.
, 2008
, “Prioritizing Equipments for Preventive Maintenance (PM) Activities Using Fuzzy Rules
,” Comput. Ind. Eng.
, 54
(2
), pp. 169
–184
. 9.
Chang
, Q.
, Xiao
, G.
, Biller
, S.
, and Li
, L.
, 2013
, “Energy Saving Opportunity Analysis of Automotive Serial Production Systems
,” IEEE Trans. Autom. Sci. Eng.
, 10
(2
), pp. 334
–342
. 10.
Gu
, X.
, Jin
, X.
, and Ni
, J.
, 2015
, “Prediction of Passive Maintenance Opportunity Windows on Bottleneck Machines in Complex Manufacturing Systems
,” ASME J. Manuf. Sci. Eng.
, 137
(3
), p. 031017
. 11.
Dekker
, R.
, 1995
, “Integrating Optimisation, Priority Setting, Planning and Combining of Maintenance Activities
,” Eur. J. Oper. Res.
, 82
(2
), pp. 225
–240
. 12.
Dekker
, R.
, and Scarf
, P. A.
, 1998
, “On the Impact of Optimisation Models in Maintenance Decision Making: the State of the Art
,” Reliab. Eng. Syst. Saf.
, 60
(2
), pp. 111
–119
.13.
Teng
, S.-H. G.
, and Ho
, S.-Y. M.
, 1996
, “Failure Mode and Effects Analysis
,” Int. J. Qual. Reliab. Manage.
, 13
(5
), pp. 8
–26
. 14.
Ding
, S.-H.
, Kamaruddin
, S.
, and Azid
, I. A.
, 2014
, “Maintenance Policy Selection Model–a Case Study in the Palm Oil Industry
,” J. Manuf. Technol. Manage.
, 25
(3
), pp. 415
–435
. 15.
Yang
, Z.
, Chang
, Q.
, Djurdjanovic
, D.
, Ni
, J.
, and Lee
, J.
, 2007
, “Maintenance Priority Assignment Utilizing on-line Production Information
,” ASME J. Manuf. Sci. Eng.
, 129
(2
), pp. 435
–446
. 16.
Birnbaum
, Z. W.
, 1968
, On The Importance of Different Components in A Multicomponent System
. Technical Report
, Washington University Seattle Lab of Statistical Research
.17.
Nguyen
, K.-A.
, Do
, P.
, and Grall
, A.
, 2014
, “Condition-Based Maintenance for Multi-Component Systems Using Importance Measure and Predictive Information
,” Int. J. Syst. Sci.: Oper. Logist.
, 1
(4
), pp. 228
–245
.18.
Si
, S.
, Liu
, M.
, Jiang
, Z.
, Jin
, T.
, and Cai
, Z.
, 2019
, “System Reliability Allocation and Optimization Based on Generalized Birnbaum Importance Measure
,” IEEE Trans. Reliab.
, 68
(3
), pp. 831
–843
. 19.
Hoffman
, M.
, Song
, E.
, Brundage
, M.
, and Kumara
, S.
, 2018
, “Condition-Based Maintenance Policy Optimization Using Genetic Algorithms and Gaussian Markov Improvement Algorithm
,” Annual Conference of the PHM Society
, Vol. 10
, Philadelphia, PA
, Prognostics and Health Management Society.20.
Sutton
, R. S.
, and Barto
, A. G.
, 2018
, Reinforcement Learning: An Introduction
, MIT Press
, Cambridge, MA
.21.
Kearns
, M.
, Mansour
, Y.
, and Ng
, A. Y.
, 2002
, “A Sparse Sampling Algorithm for Near-Optimal Planning in Large Markov Decision Processes
,” Mach. Learn.
, 49
(2
), pp. 193
–208
. 22.
Kocsis
, L.
, and Szepesvári
, C.
, 2006
, “Bandit Based Monte-Carlo Planning
,” European Conference on Machine Learning
, Berlin, Germany
, Springer
, pp. 282
–293
.23.
Browne
, C. B.
, Powley
, E.
, Whitehouse
, D.
, Lucas
, S. M.
, Cowling
, P. I.
, Rohlfshagen
, P.
, Tavener
, S.
, Perez
, D.
, Samothrakis
, S.
, and Colton
, S.
, 2012
, “A Survey of Monte Carlo Tree Search Methods
,” IEEE Trans. Comput. Intell. AI in Games
, 4
(1
), pp. 1
–43
. 24.
Hoffman
, M.
, Song
, E.
, Brundage
, M.
, and Kumara
, S.
, 2021
, “Online Improvement of Condition-Based Maintenance Policy Via Monte Carlo Tree Search
,” IEEE Trans. Autom. Sci. Eng.
, pp. 1
–12
. 25.
Kolodner
, J. L.
, 1992
, “An Introduction to Case-Based Reasoning
,” Artif. Intell. Rev.
, 6
(1
), pp. 3
–34
. 26.
Aamodt
, A.
, and Plaza
, E.
, 1994
, “Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches
,” AI Commun.
, 7
(1
), pp. 39
–59
. 27.
Bergmann
, R.
, and Wilke
, W.
, 1996
, “On the Role of Abstraction in Case-Based Reasoning
,” European Workshop on Advances in Case-Based Reasoning, Lausanne, Switzerland, Springer
, pp. 28
–43
.28.
Bengtsson
, M.
, Olsson
, E.
, Funk
, P.
, and Jackson
, M.
, 2004
, “Technical Design of Condition Based Maintenance System-A Case Study Using Sound Analysis and Case-Based Reasoning
,” 8th International Conference of Maintenance and Reliability
, Knoxville, TN
.29.
Tsai
, Y.
, 2009
, “Applying a Case-Based Reasoning Method for Fault Diagnosis During Maintenance
,” Proc. Inst. Mech. Eng., Part C: J. Mech. Eng. Sci.
, 223
(10
), pp. 2431
–2441
. 30.
Wan
, S.
, Li
, D.
, Gao
, J.
, and Li
, J.
, 2019
, “A Knowledge Based Machine Tool Maintenance Planning System Using Case-Based Reasoning Techniques
,” Rob. Comput.-Integr. Manuf.
, 58
, pp. 80
–96
. 31.
Yu
, R.
, Iung
, B.
, and Panetto
, H.
, 2003
, “A Multi-Agents Based E-Maintenance System with Case-Based Reasoning Decision Support
,” Eng. Appl. Artif. Intell.
, 16
(4
), pp. 321
–333
. 32.
Gabel
, T.
, and Riedmiller
, M.
, 2005
, “CBR For State Value Function Approximation in Reinforcement Learning
,” International Conference on Case-Based Reasoning
, Chicago, IL
, Springer
, pp. 206
–221
.33.
Varshavskii
, P.
, and Eremeev
, A.
, 2010
, “Modeling of Case-Based Reasoning in Intelligent Decision Support Systems
,” Sci. Techn. Inf. Process.
, 37
(5
), pp. 336
–345
. 34.
Li
, J.
, and Meerkov
, S. M.
, 2008
, Production Systems Engineering
, Springer Science & Business Media
, New York
.35.
Nguyen
, Q.
, Valizadegan
, H.
, and Hauskrecht
, M.
, 2011
, “Learning Classification With Auxiliary Probabilistic Information
,” 2011 IEEE 11th International Conference on Data Mining
, Vancouver, Canada
, IEEE
, pp. 477
–486
.36.
Aha
, D. W.
, 2013
, Lazy Learning
, Springer Science & Business Media
, New York
.37.
Kim
, K.-K.
, Taeho
, K.
, and Song
, E.
, 2021
, “Selection of The Most Probable Best Under Input Uncertainty
,” 2021 Winter Simulation Conference
, Phoenix, AZ
, IEEE.38.
39.
Brundage
, M. P.
, Morris
, K.
, Sexton
, T.
, Moccozet
, S.
, and Hoffman
, M.
, 2018
, “Developing Maintenance Key Performance Indicators From Maintenance Work Order Data
,” International Manufacturing Science and Engineering Conference
, College Station, TX
, Vol. 51371
, American Society of Mechanical Engineers
, p. V003T02A027
.Copyright © 2022 by ASME
You do not currently have access to this content.
