Double deep Q-network-based self-adaptive scheduling approach for smart shop floor

In the field of smart manufacturing, the data-driven scheduling approach has become an effective way to solve the smart shop floor scheduling problem with high complexity and dynamics. However, most existing approaches rely too heavily on manual supervision in implementation, resulting in poor adaptability and effectiveness in dynamic production environments. Therefore, this paper proposes a self-adaptive scheduling approach based on double deep Q-network (DDQN), which can reduce manual supervision and realize the autonomy of the whole scheduling process. In the presented approach, first, a self-adaptive scheduling framework, which forms a closed-loop optimization structure for scheduling model evaluation, generation/updating, and application, is designed. Second, the interactive learning mechanism of reinforcement learning is introduced, and the scheduling model is generated through the DDQN algorithm without manual supervision. In addition, dynamic reward function based on simulation is proposed to promote the rationality and accuracy of the reward in reinforcement learning. The effectiveness of the proposed approach is validated on a semiconductor production shop floor, and the experimental results illustrate that the proposed approach can improve the effectiveness of self-adaptive scheduling and significantly reduce the time and labour costs in the dynamic production environments.