This study proposes an improved semi-analytical approach for contact stiffness modeling of bolted joints in a machine tool system. First, nonlinear contact stress distribution within a single-bolted joint is obtained from the simulation results of finite element analysis software. Second, employing the Hertz contact theory and fractal theory, the contact stiffness model of a single asperity is formulated, affording analytical expressions for normal and tangential contact stiffnesses of a single-bolted joint by integrating multi-asperities in the contact area. Subsequently, considering two test specimens as illustrations, the mode shapes and natural frequencies of the proposed model and modal analysis tests are compared, and the influence of coupling effects between two adjacent bolts is illustrated. The maximum error in the natural frequencies of the proposed approach is < 2.73% relative to the experimental results. Finally, the measurements of frequency response functions on a box-in-box precision horizontal machine tool are conducted to demonstrate the accuracy and efficiency of the proposed model. The proposed model is highly efficient in revealing the influence of microcontact factors on the contact stiffness of bolted joints and in guiding the optimal functional design of bolt arrangements under the framework of virtual machine tools.