Digital Twin-Assisted Multi-Objective Optimization Method Based on Multi-Agent Reinforcement Learning for Five-Axis CNC Machining

Five-axis CNC machining involves strong coupling among machining quality, material removal efficiency, and operational safety, making it difficult to obtain adaptive and feasible process parameters using conventional scalar-objective optimization methods. To address this problem, this study proposes a physics-constrained multi-objective multi-agent deep deterministic policy gradient framework, termed MOMADDPG, for Pareto-oriented optimization of five-axis machining parameters. A data-calibrated digital twin simulation environment is constructed to model five-axis kinematics, tool-workpiece engagement, cutting force, chatter tendency, spindle power, tool wear, actuator bounds, and collision risk. The PHM Society 2010 milling dataset is used to calibrate the cutting force and tool wear sub-models, while five-axis motion, tool orientation variation, and engagement conditions are generated within the digital twin environment. In the proposed framework, three heterogeneous agents are assigned to quality preservation, efficiency improvement, and safety assurance, respectively. A hierarchical attention Actor is designed to enhance feature extraction under partially observable machining conditions, while vector-valued dual Critics preserve objective-specific value information. Physical constraints are handled using adaptive Lagrangian multipliers, and a Pareto archive-guided preference curriculum is introduced to improve the diversity of feasible non-dominated solutions. Simulation results show that MOMADDPG achieves a task success rate of 98% and a hypervolume value of 0.674 after training. Compared with representative baselines, including DQN, MADDPG, MAAC, and MAPPO, the proposed method provides better Pareto-front approximation, higher task feasibility, and stronger robustness under process perturbations in the data-calibrated five-axis simulation environment. The results demonstrate the potential of combining digital twins and multi-objective multi-agent reinforcement learning for safe and adaptive parameter optimization in five-axis machining simulations. Further validation on physical five-axis CNC systems is still required before industrial deployment.