9. Dynamic Selective Maintenance for Multi-state Systems Operating Multiple Consecutive Missions

For a system operating multiple consecutive missions, the condition of each component can be inspected at the end of each mission. Therefore, the selective maintenance strategy needs to be dynamically determined given the condition of components, remaining maintenance resources, and the characteristics of future missions. This chapter develops a dynamic selective maintenance model for multi-state systems operating multiple consecutive missions. The maintenance actions are dynamically determined at the beginning of each break, so as to maximize the expected number of the successes of future missions. The sequential decision problem was formulated as a Markov decision process with a mixed integer-discrete-continuous state space. To mitigate the “curse of dimensionality,” a deep reinforcement learning (DRL) algorithm based on actor-critic framework was proposed to solve the problem. A postprocess was then utilized to search for the optimal maintenance actions in a constrained large-scale action space. Two illustrative examples were given to examine the effectiveness of the proposed dynamic selective maintenance model.