Initial residual stress is the main reason causing machining deformation of the workpiece, which has been deemed as one of the most important aspects of machining quality issues. The inference of the distribution of initial residual stress inside the blank has significant meaning for machining deformation control. Due to the principle error of existing residual stress detection methods, there are still challenges in practical applications. Aiming at the detection problem of the initial residual stress field, an initial residual stress inference method by incorporating monitoring data and mechanism model is proposed in this paper. Monitoring data during machining process is used to represent the macroscopic characterization of the unbalanced residual stress, and the finite element numerical model is used as the mechanism model so as to solve the problem that the analytic mechanism model is difficult to establish; the policy gradient approach is introduced to solve the gradient descent problem of the combination of learning model and mechanism model. Finally, the initial residual stress field is obtained through iterative calculation based on the fusing method of monitoring data and mechanism model. Verification results show that the proposed inference method of initial residual stress field can accurately and effectively reflect the machining deformation in the actual machining process.