Optimisation of a Workpiece Clamping Position with Reinforcement Learning for Complex Milling Applications

Fine-tuning and optimisation of production processes in manufacturing are often conducted with the help of algorithms from the field of Operations Research (OR) or directly by human experts. Machine Learning (ML) methods demonstrate outstanding results in tackling optimisation tasks within the research field referred to as Neural Combinatorial Optimisation (NCO). This opens multiple opportunities in manufacturing for learning-based optimisation solutions. In this work, we show a successful application of Reinforcement Learning (RL) to the task of workpiece (WP) clamping position and orientation optimisation for milling processes. A carefully selected clamping position and orientation of a WP are essential for minimising machine tool wear and energy consumption. With the example of 3- and 5-axis milling, we demonstrate that a trained RL agent can successfully find a near-optimal orientation and positioning for new, previously unseen WPs. The achieved solution quality is comparable to alternative optimisation solutions relying on Simulated Annealing (SA) and Genetic Algorithms (GA) while requiring orders of magnitude fewer optimisation iterations.